957: Add manual locking to `ASYNCIO` for transacted writes r=Learath2 a=heinrich5991
Previously, e. g. it was possible that newlines are separated from the
printed line in `logger_file`.
869: Complete rework of the ghost and race recorder r=Learath2 a=Redix
This PR fixes a lot of issues with the ghost and race recorder, introduces a new ghost file format and implements some pretty useful features.
**A quick overview:**
- When you cross the start line multiple times, both (ghost and recorder) will only restart if it is a non-solo server
- If available, both will use the race timer to recognize the start instead of searching for the start line
- Fastcap support for both
- The recorder immediately starts when the Tee spawns, so the preparation steps (which are quite important for speedruns) will be included in the demo. If you do not cross the start-line within 20 seconds, it will stop the demo until you really start.
- Better ghost menu (colors, reloading, deleting and saving ghosts)
- The ghosts are more resistant against lags (old ones were sometimes completely useless due to small lags)
- New ghosts files are significantly smaller
- Cleanup, bugfixes..
**About the new ghost format (version 4/5):**
This format is used by Teerace for over a year now. The code for handling the binary files was moved to the engine. It includes an auto updater which creates a backup of all ghosts and converts them to the new format afterwards. The major differences from the format used by DDNet right now (version 2), are the ability to store multiple types of data, the usage of delta-encoding and a more portable header.
For whatever reason, the ghost stores values for every predicted tick, but without tick information, so lost snapshots can make them unusable. The new code uses the original values from the snapshots including ticks but it can also handle the old ones without. Since hardly any server uses the high bandwidth option this practically reduces the file size.
Like the demo recorder the ghost recorder directly stores the data to a file (every 50 snapshots) instead of writing the whole file at the end of the race. Indeed this can be changed with only a few lines if the old behavior is preferred.
The updater can handle version 2 (DDNet) and 3 (old teerace format, only slightly different from version 2) files. The updating already happens when the files are scanned for generating the list in the menu and not only when you activate them. The change from version 4 to 5 was only needed due to a bug in the implementation, the ghost loader can read both.
Some numbers about the file size: (map: hotrun, both about 30 seconds)
- Old ghost: 30.4 kB (converted: 10.7 kB)
- New ghost: 5.4 kB
**One thing about the race recorder:**
The old implementation compared the new file only with the first file it found for the particular map. The new one compares with all related demos and deletes them possibly, so that only the best demo is left. Since DDNet can also store the demos without name, this might also delete demos from other players, that you might have in your directory.
To prevent this I at least check whether the demo contains the player name if `cl_demo_name` is on.
In my opinion the better solution would be to remove `cl_demo_name` and always use the player name.
Also annotate `dbg_msg`, `str_format` and `str_timestamp_ex` so that the
compiler can determine whether the format strings are correct.
Fix the compiler warnings generated by these extra warnings -- some of
them were security issues.
This also fixes a couple of potential buffer overflows. The affected
code is still buggy (doesn't do the right thing on potential buffer
overflow), but at least it won't result in crashes/out of bound writes
anymore.
This means that we have a reliable and fast way to query for extended info,
while also not wasting network bandwidth.
The protocol is designed to be extensible, there's four bytes space for
encoding more request types (currently zeroed), and there's one string in each
response packet and one string for each player available (currently the empty
string).
The protocol itself has no problems with more than 64 players, although the
current client implementation will drop the player info after the 64th player,
because it uses a static array for storage.
Also fixes#130, the player list is just sorted each time new player info
arrives.
See UTS#39 "Unicode Security Mechanisms":
http://www.unicode.org/reports/tr39/
This means that characters with accents or other things around them are
now considered the confusable with the base character.
Fixes#557. Fixes#575.
This not only makes `str_utf8_decode` a compliant parser, but also fixes an
issue where overlong encodings led to undetected line breaks.
Implemented as per https://encoding.spec.whatwg.org/#utf-8-decoder.
Audited the following files, checking whether the new semantics work with the
existing code:
- src/base/system.c
- src/engine/client/text.cpp
- src/game/client/components/chat.cpp
- src/game/server/gamecontext.cpp
They all check whether the string is at its end separately.
