Add strict validation for `StrToInts` function. Because this function should only be used with trusted internal strings, assertions are added to ensure that the string is not truncated. Some buffer sizes are adjusted accordingly, so truncation cannot happen.
Add less strict validation for `IntsToStr` function. An additional argument specifying the size of the output buffer is added to assert that the size of the output buffer is sufficient. However, because this function is used to decode data sent by the server and read from maps and ghosts, invalid input data should never result in crashes or invalid UTF-8 strings. The function will now unpack an empty string and return `false`, if the string contains invalid UTF-8.
The inline definition of the functions is not wanted, because it requires adding a `system.h` include in `gamecore.h`. Therefore the tools now have to depend on game-shared, which previously only worked because the functions were inline.
Tests are added to ensure the function still behaves the same as before for valid inputs and correctly handles invalid inputs.
Add `IJob::Abortable(bool)` function which jobs can call to specify whether they can be aborted. Jobs are not abortable per default. Abortable jobs may have their state set to `IJob::STATE_ABORTED` at any point if the job was aborted. The job state should be checked periodically in the `IJob::Run` function and the job should terminate at the earliest, safe opportunity when aborted. Scheduled jobs which are not abortable are guaranteed to fully complete before the job pool is shut down. However, if the job pool is already shutting down, no additional jobs will be enqueue anymore and abortable jobs will immediately be aborted.
In particular, the sound loading, community icon loading, master chooser and host lookup jobs are specified as being abortable. Conversely, the jobs saving replay demos, editor maps and screenshots are expected to finish before the client is shut down.
When the client is quitting/restarting, it will now disconnect from the current server first, before saving the config, to ensure that any actions that happen on disconnect (demo recorders being stopped etc.) happen first. The shutdown message is rendered before disconnecting and waiting for background jobs to finish.
The HTTP client is now initialized later during server launch, after the network initialization. Error handling is added and the server stops if the HTTP client could not be initialized, same as the client.
The `RunBlocking` functions are removed, as they are not used anymore after curl-multi was added.
The function `IJob::Status` is renamed to `State` and `IJob::STATE_PENDING` is renamed to `STATE_QUEUED` for consistency with naming of the HTTP client.
The member variables of the engine interface are encapsulated and the `jobs.h` include is removed from `engine.h`, which removes transitive includes of `system.h`.
Documentation for all job and job pool API is added.
Files were not being truncated on Windows anymore when using `io_open` with `IOFLAG_WRITE` due to a regression from #7254. Instead, the existing file contents were kept and the file pointer was set to the beginning of the file.
This caused broken demo files to be created (#7349) when recording a shorter demo with the same filename as an existing longer demo. It also caused the map tools to produce maps with additional junk data at the end, if an existing map is overridden by a smaller map.
This is fixed by using the creation disposition `CREATE_ALWAYS` instead of `OPEN_EXISTING` with [`CreateFileW`](https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilew), which ensures that a file is always created and truncated.
A regression test case is added, which fails without this change.
Add stricter error handling when converting between UTF-16 (wide characters) and UTF-8 (multi-byte) on Windows.
The `windows_wide_to_utf8` function now returns an `std::optional`, which will be empty if the argument contains invalid UTF-16. Files/folders with names containing invalid UTF-16 are now ignored on Windows. It was previously not possible to use these files either, as converting their names to UTF-8 changed the invalid codepoints to unicode replacement characters.
The `windows_utf8_to_wide` function now fails with an assertion error if the argument contains invalid UTF-8, as this should never happen.
Closes#7486.
Unify all code for name bans in new class `CNameBans` in the existing `name_ban.cpp/h` files. The previously global function `IsNameBanned` is now the member function `CNameBans::IsBanned`. The existing name ban tests are extended for the `CNameBans` class.
Move `CNameBan` constructor definition to source file to avoid including `system.h` in the header file. Use `bool` instead of `int` for `m_IsSubstring`. Reorder `CNameBan` constructor arguments and remove unnecessary default value.
Initialize nontrivial types with a constructor instead. Make the
compiler aware that some of our constructors are indeed trivial.
This allows `mem_zero` calls to actually always zero the memory.
Partially replaces #5690.
POD types are just memset. Other types are either destructed if not
trivial and/or constructed if not trivial. Types need to have a default
constructor.
Virtual classes can be mem_zeroed only if they already have been
constructed, otherwise it is UB.
Replace usages of platform specific `lock_*` functions with `std::mutex` through the wrapper class `CLock`. Move lock classes to `base/lock.h`.
The `CLock` wrapper class is only necessary because the clang thread-safety attributes are not available for `std::mutex` except when explicitly using libc++.
Round to nearest integer instead of truncating in `f2fx` to ensure correct round-trip with `fx2f`.
Add test to ensure correct round-trip with maximum `0.0005f` absolute error.
The host lookup job and the engine interface are independent so they are moved to separate files.
The include of `engine.h` in `client.h` is therefore unnecessary and other includes also had to be adjusted because of this.
The variable `m_VersionServeraddr` is unused and therefore removed. The host lookup job is currently not used on the client-side.
Simplify the usage of datafile reader and writer by adding utility functions to read and write zero-terminated UTF-8 strings.
Improve validation of string data read from datafiles. It is ensure that string data is null-terminated, has no internal NUL-characters and is valid UTF-8.
Fix loading of external sounds in the editor. The wrong path variable was being used, so the sound files would not be loaded from correct folder.
Add tests for new datafile reader/writer functions.
vanilla 0.6 joins weren't recorded. Also after map change, for the
existing players these Join chunks were missing. Just add these
message for existing players now and add OnClientEngineJoin in all
code paths with clients connecting.
Add `str_utf8_offset_bytes_to_chars` and `str_utf8_offset_chars_to_bytes` functions to base system to convert between byte and UTF-8 character offsets in UTF-8 strings.
Previously, this was separately implemented in the textrender and in the lineinput helper.
These textrender functions are entirely replaced by the new functions:
- `ITextRender::SelectionToUTF8OffSets` (by `str_utf8_offset_chars_to_bytes`)
- `ITextRender::UTF8OffToDecodedOff` (by `str_utf8_offset_bytes_to_chars`)
- `ITextRender::DecodedOffToUTF8Off` (by `str_utf8_offset_chars_to_bytes`)
These lineinput helper functions are reimplemented using the new functions:
- `CLineInput::OffsetFromActualToDisplay` (uses `str_utf8_offset_bytes_to_chars`)
- `CLineInput::OffsetFromDisplayToActual` (uses `str_utf8_offset_chars_to_bytes`)
This still works
$ ./DDNet "connect 127.0.0.1"
But now also this works
$ ./DDNet "connect tw-0.6+udp://ger10.ddnet.org:8303"
2023-07-16 14:07:50 I engine: running on unix-linux-amd64
2023-07-16 14:07:50 I client: starting...
2023-07-16 14:07:50 I client: version 17.1.1 on linux amd64
2023-07-16 14:07:50 I client: git revision hash: 7f100e2620
2023-07-16 14:07:50 I client: connecting to 'tw-0.6+udp://ger10.ddnet.org:8303'
2023-07-16 14:07:50 I host_lookup: host='ger10.ddnet.org' port=8303 3
2023-07-16 14:07:51 I client: connected, sending info
Port the `CJsonWriter` utility class from upstream, which makes outputting correct JSON easier.
Add `CJsonWriter` as an abstract class that can write to different outputs. Two implementations `CJsonFileWriter` (writes to a file) and `CJsonStringWriter` (writes to an `std::string`) are added. Upstream `CJsonWriter` can only write to files.
The same tests are added for both implementations. Duplicate code is avoided by using typed tests with two separate test fixtures.
Typed tests have test names like "TestName/0" and "TestName/1", which would result in invalid filenames. Replace the string after the first slash with the name of the typed test and use hyphen instead of slash.
Add optional `end` output parameter to `str_utf8_find_nocase` which is set to a pointer into the haystack after the matched string.
This is necessary because there are pairs of matching upper case and lower case Unicode characters with different byte length (e.g. both `I` and `İ` map to `i`), so the byte length of the string matched in the haystack may not be identical to the length of the needle string.
ChillerDragon
