Converting wide-char (UTF-16) to multi-byte (UTF-8) takes 1-2 wide-chars from the input and transforms them to 1-4 bytes, so having `char` and `WCHAR` buffers with equal static lengths means that this function can fail due to insufficient buffer sizes when the user has folder or files with very long names (many unicode codepoints).
Therefore checks are added that allow only the error `ERROR_INSUFFICIENT_BUFFER` when `WideCharToMultiByte` fails, which is expected on very long paths, as these would also lead to further errors later in the code. The respective functions will now fail with a return and ignore files that have too long names.
This could only completely be fixed using dynamically sized buffers for all paths, which seems like too much work and overhead.
Other errors are not expected and hence caught by the assertions, as those would indicate programming errors like wrong arguments being passed.
Reference: https://learn.microsoft.com/en-us/windows/win32/api/stringapiset/nf-stringapiset-widechartomultibyte
The second buffer was previously used when calling `fs_is_dir` explicity, but it has become from obsolete from #4657 by using the flag `FILE_ATTRIBUTE_DIRECTORY` instead.
Extract `windows_format_system_message` and allocate appropriate buffer for the formatted system error messages, so messages of any length can be displayed.
The flag `FORMAT_MESSAGE_ALLOCATE_BUFFER` is used so `FormatMessageW` allocates the buffer which must later be freed with `LocalFree`.
The flag `FORMAT_MESSAGE_MAX_WIDTH_MASK` is also added so the formatted message will not contain any line breaks at the end, which would make log messages less readable.
Reference: https://learn.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-formatmessagew
Converting multi-byte (UTF-8) to wide-char (UTF-16) take 1-4 bytes from the input and transforms them to 1-2 wide-chars, so having `char` and `WCHAR` buffers with equal lengths should mean that this function can't fail due to insufficient buffer sizes, unless a path that's already too long for Windows is being used internally.
Other errors are also not expected, as those would indicate programming errors like wrong arguments being passed.
The maximum length for paths in the Win32 API is 260 characters. This limitation can be lifted in Windows 10, but it must be done explicitly (opt-in) both in the Windows Registry by the user and in the application's manifest by us.
References:
- https://learn.microsoft.com/en-us/windows/win32/api/stringapiset/nf-stringapiset-multibytetowidechar
- https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=registry
By moving the calling convention `APIENTRY`, which expands to `__stdcall`, inside the parenthesis.
```
Severity Code Description Project File Line Suppression State
Error C2143 syntax error: missing ')' before '(' engine-shared src\base\system.cpp 4242
Warning C4229 anachronism used: modifiers on data are ignored engine-shared src\base\system.cpp 4242
Error C2059 syntax error: ')' engine-shared src\base\system.cpp 4242
Error C2059 syntax error: ')' engine-shared src\base\system.cpp 4242
Error C3536 'exception_log_file_path_func': cannot be used before it is initialized engine-shared src\base\system.cpp 4246
Error C2446 '==': no conversion from 'nullptr' to 'int' engine-shared src\base\system.cpp 4246
Error C2064 term does not evaluate to a function taking 1 arguments engine-shared src\base\system.cpp 4249
```
5942: Support unicode with ExcHndl, use upstream module offsets, handle errors r=def- a=Robyt3
Update Dr. Mingw (ExcHndl) to 0.9.8.
Use the new `ExcHndlSetLogFileNameW` function to set the exception log file name using wide characters, to support paths containing unicode.
It's not necessary to call `ExcHndlInit` explicitly after loading `exchndl.dll`, as the `DllMain` will already initialize the exception handler when the DLL is loaded. Module offsets are supported by upstream ExcHndl now, so we don't need to provide our own version that supplies the module offset to `ExcHndlInit` anymore. Upstream ExcHndl will also resolve the source code lines for addresses automatically, when the executable is build with debug information.
Handle the cases that the exception handling module cannot be loaded and that the `ExcHndlSetLogFileNameW` function cannot be found in the module.
Update `scripts/parse_drmingw.sh`:
- Parse both old and new module offsets.
- Use tabs instead of spaces consistently.
- Reset the ANSI color after printing colored messages.
Closes#5877.
Needs https://github.com/ddnet/ddnet-libs/pull/34.
Example crash logs:
- [crash_debug_old.RTP.txt](https://github.com/ddnet/ddnet/files/9768008/crash_debug_old.RTP.txt)
- [crash_debug_new.RTP.txt](https://github.com/ddnet/ddnet/files/9768011/crash_debug_new.RTP.txt)
- [crash_release_old.RTP.txt](https://github.com/ddnet/ddnet/files/9768010/crash_release_old.RTP.txt)
- [crash_release_new.RTP.txt](https://github.com/ddnet/ddnet/files/9768009/crash_release_new.RTP.txt)
## Checklist
- [X] Tested the change ingame
- [ ] Provided screenshots if it is a visual change
- [ ] Tested in combination with possibly related configuration options
- [ ] Written a unit test (especially base/) or added coverage to integration test
- [ ] Considered possible null pointers and out of bounds array indexing
- [ ] Changed no physics that affect existing maps
- [ ] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssanitizer--undefinedbehavioursanitizer-or-valgrinds-memcheck) (optional)
Co-authored-by: Robert Müller <robytemueller@gmail.com>
Update Dr. Mingw (ExcHndl) to 0.9.8.
Use the new `ExcHndlSetLogFileNameW` function to set the exception log file name using wide characters, to support paths containing unicode.
It's not necessary to call `ExcHndlInit` explicitly after loading `exchndl.dll`, as the `DllMain` will already initialize the exception handler when the DLL is loaded.
Module offsets are supported by upstream ExcHndl now, so we don't need to provide our own version that supplies the module offset to `ExcHndlInit` anymore.
Upstream ExcHndl will also resolve the source code lines for addresses automatically, when the executable is build with debug information.
Handle the cases that the exception handling module cannot be loaded and that the `ExcHndlSetLogFileNameW` function cannot be found in the module.
Update `scripts/parse_drmingw.sh`:
- Parse both old and new module offsets.
- Use tabs instead of spaces consistently.
- Reset the ANSI color after printing colored messages.
- Retrieve version information of `kernel32.dll` without providing a fixed path to the Windows directory, to handle the case that Windows is not installed in the default location `C:\Windows`. This works because `GetFileVersionInfoSizeW` and `GetFileVersionInfoW` use the search sequence defined by `LoadLibrary`, so they will find the DLL without the path explicitly being specified. The version is retrieved from `kernel32.dll` instead of `user32.dll`, as the former is more commonly used for this purpose.
- Use the unicode (wide character) functions consistently instead of the ANSI functions.
- Use the correct format specifier `%hu` for `unsigned short` (i.e. `WORD` in the Windows API).
References:
- https://learn.microsoft.com/en-us/windows/win32/api/winver/nf-winver-getfileversioninfosizew
- https://learn.microsoft.com/en-us/windows/win32/api/winver/nf-winver-getfileversioninfow
- https://stackoverflow.com/a/44672263/1708371
Update the `str_skip_to_whitespace(_const)` functions according to their documentation, which already stated that `\r` was also considered as whitespace.
This changes `str_comp_filenames` so it sorts filenames case insensitive while also comparing digits characters as numbers.
This makes filename sorting consistent with the behavior in Windows Explorer.
- Change argument and return value to `const char *`, as the string is not modified by this function.
- Use our own `str_isspace` instead of standard library `isspace`.
- Always trim leading whitespace to correctly handle inputs with leading whitespace.
For more consistent usage of the Windows API. `_wgetcwd` seems to be a wrapper around `GetCurrentDirectoryW` anyway, at least in the [Wine API](https://source.winehq.org/ident?_i=_wgetcwd).
Pass the correct size of the wide char buffer instead of passing the size of the output parameter buffer.
Remove a nullptr check. The argument should never be null and we don't check for null arguments anywhere else.
For more consistent usage of the Windows API. `_wchdir` seems to be a wrapper around `SetCurrentDirectoryW` anyway, at least in the [Wine API](https://source.winehq.org/ident?_i=_wchdir).
For more consistent usage of the Windows API. `_wmkdir` seems to be a wrapper around `CreateDirectoryW` anyway, at least in the [Wine API](https://source.winehq.org/ident?_i=_wmkdir).
Previously, the socket addresses were truncated as the `msg_namelen`
field is both input **and** output: After receiving an IPv4 packet, the
socket address field would be too short for an IPv6 address.
5760: Fix variable shadow warnings with MinGW r=def- a=Robyt3
```
ddnet/src/base/system.cpp: In function 'int byteval(const char*, unsigned char*)':
ddnet/src/base/system.cpp:3044:32: warning: declaration of 'byte' shadows a global declaration [-Wshadow]
3044 | static int byteval(const char *byte, unsigned char *dst)
| ~~~~~~~~~~~~^~~~
In file included from msys64/mingw64/include/objbase.h:8,
from ddnet/src/base/system.cpp:68:
msys64/mingw64/include/rpcndr.h:63:25: note: shadowed declaration is here
63 | typedef unsigned char byte;
| ^~~~
ddnet/src/base/system.cpp: In function 'unsigned char str_byte_next(const char**)':
ddnet/src/base/system.cpp:3557:23: warning: declaration of 'byte' shadows a global declaration [-Wshadow]
3557 | unsigned char byte = **ptr;
| ^~~~
msys64/mingw64/include/rpcndr.h:63:25: note: shadowed declaration is here
63 | typedef unsigned char byte;
| ^~~~
ddnet/src/base/system.cpp: In function 'int str_utf8_decode(const char**)':
ddnet/src/base/system.cpp:3577:31: warning: declaration of 'byte' shadows a global declaration [-Wshadow]
3577 | unsigned char byte = str_byte_next(ptr);
| ^~~~
msys64/mingw64/include/rpcndr.h:63:25: note: shadowed declaration is here
63 | typedef unsigned char byte;
| ^~~~
```
<!-- What is the motivation for the changes of this pull request -->
## Checklist
- [ ] Tested the change ingame
- [ ] Provided screenshots if it is a visual change
- [ ] Tested in combination with possibly related configuration options
- [ ] Written a unit test (especially base/) or added coverage to integration test
- [ ] Considered possible null pointers and out of bounds array indexing
- [ ] Changed no physics that affect existing maps
- [ ] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssanitizer--undefinedbehavioursanitizer-or-valgrinds-memcheck) (optional)
Co-authored-by: Robert Müller <robytemueller@gmail.com>
5754: Auto refresh skins when changing related settings r=def- a=Jupeyy
motivation: downloaded skins aren't resettet, e.g. if they failed before. config change -> resets everything
## Checklist
- [x] Tested the change ingame
- [ ] Provided screenshots if it is a visual change
- [ ] Tested in combination with possibly related configuration options
- [ ] Written a unit test (especially base/) or added coverage to integration test
- [ ] Considered possible null pointers and out of bounds array indexing
- [ ] Changed no physics that affect existing maps
- [ ] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssanitizer--undefinedbehavioursanitizer-or-valgrinds-memcheck) (optional)
5757: Make file link absolute, add `fs_is_relative_path` r=def- a=Robyt3
This fixes links not opening for relative paths, as links like `file://temp/skins` cannot be resolved by the shell.
Closes#5746.
## Checklist
- [X] Tested the change ingame (only on Windows)
- [ ] Provided screenshots if it is a visual change
- [ ] Tested in combination with possibly related configuration options
- [ ] Written a unit test (especially base/) or added coverage to integration test
- [ ] Considered possible null pointers and out of bounds array indexing
- [ ] Changed no physics that affect existing maps
- [ ] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssanitizer--undefinedbehavioursanitizer-or-valgrinds-memcheck) (optional)
Co-authored-by: Jupeyy <jupjopjap@gmail.com>
Co-authored-by: Robert Müller <robytemueller@gmail.com>
On Windows, the check was incorrectly using logical or instead of bitwise or. Checking for the flag `FILE_ATTRIBUTE_REPARSE_POINT` is not necessary, as the `FILE_ATTRIBUTE_DIRECTORY` will correctly be set when the target of a symbolic link is a directory. This otherwise causes symbolic links to files to be incorrectly handled as directories.
On Linux, the minor performance optimization of using `entry->d_type` is reverted and `fs_is_dir` is used instead. This internally uses `stat`, which correctly returns the attributes for the symbolic link targets.
5205: Allow multiple addresses per server in the serverbrowser r=def- a=heinrich5991
Support is incomplete for `leak_ip_address_to_all_servers` (will only
ping the first address of each server) and for the `leak_ip` setting
(which will also only ping the first address of each server).
Fixes#5158.
## Checklist
- [x] Tested the change ingame
- [ ] Provided screenshots if it is a visual change
- [ ] Tested in combination with possibly related configuration options
- [ ] Written a unit test if it works standalone, system.c especially
- [ ] Considered possible null pointers and out of bounds array indexing
- [ ] Changed no physics that affect existing maps
- [ ] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssanitizer--undefinedbehavioursanitizer-or-valgrinds-memcheck) (optional)
Co-authored-by: heinrich5991 <heinrich5991@gmail.com>
- `io_read_all` reads all bytes from a file handle into a new buffer. It should only need one allocation per file in cases where the actual file size matches the expected file size. Otherwise it falls back to doubling the buffer size if the actual file size is larger than expected to avoid TOCTOU problems.
- `io_read_all_str` reads all bytes from a file handle into a new buffer and also ensures that the buffer is null-terminated and contains no other null-characters.
- `mem_has_null` is a utility used by `io_read_all_str` to ensure that no null-characters exist in the bytes read from the file.
It didn't have a clear role, it just acted as a distinguisher between
two functions with the same name.
Rename `tw::time_get` to `time_get_nanoseconds` and delete the old
`time_get_nanoseconds`. Move `CCmdlineFix` and the typed
`net_socket_read_wait` function to the global namespace.
