Port the `CJsonWriter` utility class from upstream, which makes outputting correct JSON easier.
Add `CJsonWriter` as an abstract class that can write to different outputs. Two implementations `CJsonFileWriter` (writes to a file) and `CJsonStringWriter` (writes to an `std::string`) are added. Upstream `CJsonWriter` can only write to files.
The same tests are added for both implementations. Duplicate code is avoided by using typed tests with two separate test fixtures.
Port the line input (UI edit boxes, chat, console) and Input Method Editor (IME) support from upstream. Closes#4397.
General
------------------------------
Fix issues with the text input. Closes#4346. Closes#4524.
Word skipping (when holding Ctrl) is overhauled to be consistent with the Windows / Firefox experience that I took as reference.
Improve usability by not blinking (i.e. always rendering) the caret shortly after is has been moved.
UI text input
------------------------------
Fix inconsistent mouse-based left and right scrolling (closes#4347).
Support smooth left and right scrolling.
Chat
------------------------------
Support keyboard-based text selection of the chat input.
Mouse-based selection could be support in the future when we decide to add something like an ingame UI cursor.
Support smooth up and down scrolling of the chat input, removing the old hack that offsets the input string to simulate scrolling.
Console
------------------------------
Also support mouse-based text selection of the command input.
Only text from either the command input or the console log can be selected at the same time. This ensures that Ctrl+C will always copy the text that is currently visually selected in the console.
Check for Ctrl+C input event in event handler instead of in render function, to hopefully fix the issue that copying does not work sometimes (closes#5974 until further notice).
When Ctrl+C is used to copy text from the console log, the selection is cleared. This should make it more clear when text was copied from the log.
Fix an issue that was preventing the console log selection from being cleared, when all log lines are selected.
Remove Ctrl+A/E hotkeys that move cursor to beginning/end respectively. Ctrl+A now selectes all text like for all other inputs. Home and End keys can still be used to go the beginning and end.
Remove Ctrl+U/K hotkeys that clear everything before/after the cursor respectively. Hold shift and use Home/End to select everything instead.
IME support
------------------------------
Render list of IME candidates in the client on Windows, so the candidate list can also be viewed in fullscreen mode. There is no API available to retrieve a candidate list on the other operating systems.
Improve composition rendering by underlining the composition text instead of putting it in square brackets.
Track active input globally to properly activate and deactivate IME through the SDL functions.
Closes#1030. Closes#1008.
Password rendering
------------------------------
Fix rendering of passwords containing unicode. Instead of rendering one star character for each UTF-8 `char`, render on star for every unicode codepoint.
Show the composition text also for passwords. Without seeing the composition text it's hard to type a password containing those characters. The candidate window exposes the composition anyway. If you don't want to expose your password this way, e.g. while streaming, you could:
1. Use a latin password and switch off the IME for the password input with the IME hotkey.
2. Blank your screen with an external program while you are streaming and entering passwords.
3. Use binds to authenticate in rcon or to set the server browser password.
Refactoring
------------------------------
Move all text input logic and general rendering to `CLineInput`.
A `CLineInput` is associated with a particular `char` buffer given as a pointer either in the constructor or with `SetBuffer`. The maximum byte size of the buffer must also be specified. The maximum length in unicode codepoints can also be specified separately (e.g. on upstream, name are limited by the number of unicode codepoints instead).
Add `CLineInputBuffered`, which is a `CLineInput` that own a `char` buffer of a fixed size, which is specified as a template argument. As `CLineInput` does not own a buffer anymore, this reduces duplicate code for line inputs that need their own buffer.
Add `CLineInputNumber` which has additional convenience functions to consider the text as an `int` or `float`, to reduce duplicate code in those cases. In the future we could also add an input filter function so that only numbers can be entered in the number input.
Add `CLineInput::SetClipboardLineCallback` to handle the case that multiple lines of text are pasted into a lineinput. This reduces duplicate code, as this behavior was previously implemented separately for chat and console. The behavior is also fixed to be consistent with the console on Windows, so the first line being pasted edits the current input text and then sends it instead of being sent on its own without the existing input text.
Add `CalcFontSizeAndBoundingBox` to UI to reduce duplicate code. Expose `CalcAlignedCursorPos` as static member function to reuse it for line input.
Dispatch input events to UI inputs through the event handler instead of storing them in a duplicate buffer.
Use `size_t` for line input cursor position, length etc. and for `str_utf8_stats`.
Add `IButtonColorFunction` to UI to describe a functions that defines colors for the Default, Active and Hovered states of UI elements. Add some default button color functions. Use button color function to reduce duplicate code in scrollbar rendering.
Use `vec2` instead of two `floats` to represent the mouse positions in the text renderer.
Remove `CaretPosition` again, as it does not calculate the correct Y position near line breaks due to the wrapping being different when not rendering the entire string. Instead, calculate the exact caret position when rending a text container and store the caret position in the text cursor for later use.
IME usage guide (Windows)
------------------------------
1. Install the respective language and the Microsoft-IME keyboard (e.g. for Chinese, Japanese or Korean).
2. Launch the game (or a text editor to first try out the IME). Note that Windows may track the input language separately for every application. You can change this in the Windows input settings so the input language is changed globally.
2. Switch the input language using the hotkey Windows+Space or another hotkey that you configured in the Windows input settings (Alt+Shift is the default, but you should consider disabling it, to avoid accidentally changing the input language while playing).
3. Switch from Latin/English input mode to the respective asian input mode.
- Chinese: Use Ctrl+Space to switch between English and Chinese input mode. You can change this hotkey in the IME's settings.
- Japanese: Use Ctrl+Space to switch between Alphanumeric and Hiragana/Katakana input mode. You can change this hotkey in the IME's settings.
- Korean: Use Right Alt to switch between English and Hangul input mode. You cannot change this hotkey as of yet.
- Note that the input mode is also tracked per application, but there is no setting to change this behavior as far as I know, so you'll need to switch for every application separately.
4. Start typing. The underlined text is the current composition text. While a composition is active, you can only edit the composition text. Confirm the composition with Space or by selecting a candidate from the candidate list with the arrow keys. Cancel the composition with Escape or by using Backspace to delete the composition text. Note that not all languages offer a candidate list.
SDL version-specific issues
------------------------------
- 2.26.5, 2.24.2, 2.0.22: IME candidates work. But there are minor bugs when moving the composition cursor.
- 2.0.18, 2.0.20: IME candidates work.
- 2.0.16 (our current version): IME candidates cannot be determined with Windows API. Windows tries to draw the composition window like before, so this does not work in fullscreen mode.
- 2.0.8 (upstream 0.7): IME candidates work. But this SDL version is too old for us.
Different target names causes long (unjustified) rebuild on the name changed.
The constant names are also better for the work with CMake targets, e.g.
one can use the same 'game-server' target across all custom setups.
Replace existing listbox implementations (`CMenus::UiDoListbox*` and `HandleListInputs` functions) with `CListBox` from upstream.
Reimplement additional feature that was already present in ddnet: page up/down, home and end key handling.
Affects the following lists:
- server browser
- server browser scoreboard
- server browser friends
- country / region selection popup (server browser filter)
- player skin list
- player country / region list
- theme list
- assets list
- graphics resolutions list
- dropdown menus (e.g. graphics fullscreen mode)
- ingame player list
- vote options list
- kick/specvote lists
- ghost list
- language list (in settings and in popup on first launch)
- demo browser
- editor file browser (saving, loading, adding images / sounds)
- The search / filename input is also improved so navigating a filtered list works correctly by porting the logic from upstream.
There are minor changes to the visual appearance of some lists, due to changed margins.
The vertical alignment of some list item texts is improved so the text is centered vertically.
/usr/include/c++/8/bits/stl_vector.h: In member function ‘void CCommandProcessorFragment_Vulkan::CleanupVulkan() [with bool IsLastCleanup = true]’:
/usr/include/c++/8/bits/stl_vector.h:1085:4: note: parameter passing for argument of type ‘__gnu_cxx::__normal_iterator<CCommandProcessorFragment_Vulkan::SMemoryBlock<2>*, std::vector<CCommandProcessorFragment_Vulkan::SMemoryBlock<2>, std::allocator<CCommandProcessorFragment_Vulkan::SMemoryBlock<2> > > >’ changed in GCC 7.1
_M_realloc_insert(end(), __x);
^~~~~~~~~~~~~~~~~
5599: Add support for Rust code in DDNet r=def- a=heinrich5991
The glue is done using the [cxx crate](https://cxx.rs/) on the Rust side.
As a proof-of-concept, only a small console command (`rust_version`) printing the currently used Rust version was added.
You can generate and open the Rust documentation using `DDNET_TEST_NO_LINK=1 cargo doc --open`.
You can run the Rust tests using `cmake --build <build dir> --target run_rust_tests`, they're automatically included in the `run_tests` target as well.
Rust tests don't work on Windows in debug mode on Windows because Rust cannot currently link with the debug version of the C stdlib on Windows: https://github.com/rust-lang/rust/issues/39016.
---
The stuff in `src/rust-bridge` is generated using
```
cxxbridge src/engine/shared/rust_version.rs --output src/rust-bridge/engine/shared/rust_version.cpp --output src/rust-bridge/engine/shared/rust_version.h
cxxbridge src/engine/console.rs --output src/rust-bridge/cpp/console.cpp --output src/rust-bridge/cpp/console.h
```
Co-authored-by: heinrich5991 <heinrich5991@gmail.com>
6002: Explain -DANTIBOT=ON a bit better (fixes#6001) r=heinrich5991 a=def-
<!-- What is the motivation for the changes of this pull request? -->
<!-- Note that builds and other checks will be run for your change. Don't feel intimidated by failures in some of the checks. If you can't resolve them yourself, experienced devs can also resolve them before merging your pull request. -->
## Checklist
- [ ] Tested the change ingame
- [ ] Provided screenshots if it is a visual change
- [ ] Tested in combination with possibly related configuration options
- [ ] Written a unit test (especially base/) or added coverage to integration test
- [ ] Considered possible null pointers and out of bounds array indexing
- [ ] Changed no physics that affect existing maps
- [ ] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssanitizer--undefinedbehavioursanitizer-or-valgrinds-memcheck) (optional)
6003: Fix height and margin of sound and image popups r=def- a=Robyt3
## Checklist
- [X] Tested the change ingame
- [ ] Provided screenshots if it is a visual change
- [ ] Tested in combination with possibly related configuration options
- [ ] Written a unit test (especially base/) or added coverage to integration test
- [ ] Considered possible null pointers and out of bounds array indexing
- [ ] Changed no physics that affect existing maps
- [ ] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssanitizer--undefinedbehavioursanitizer-or-valgrinds-memcheck) (optional)
Co-authored-by: Dennis Felsing <dennis@felsin9.de>
Co-authored-by: Robert Müller <robytemueller@gmail.com>
The glue is done using the [cxx crate](https://cxx.rs/) on the Rust
side.
As a proof-of-concept, only a small console command (`rust_version`)
printing the currently used Rust version was added.
You can generate and open the Rust documentation using
`DDNET_TEST_NO_LINK=1 cargo doc --open`.
You can run the Rust tests using `cmake --build <build dir> --target
run_rust_tests`, they're automatically included in the `run_tests`
target as well.
Rust tests don't work on Windows in debug mode on Windows because Rust
cannot currently link with the debug version of the C stdlib on Windows:
https://github.com/rust-lang/rust/issues/39016.
---
The stuff in `src/rust-bridge` is generated using
```
cxxbridge src/engine/shared/rust_version.rs --output src/rust-bridge/engine/shared/rust_version.cpp --output src/rust-bridge/engine/shared/rust_version.h
cxxbridge src/engine/console.rs --output src/rust-bridge/cpp/console.cpp --output src/rust-bridge/cpp/console.h
```
Previously warnings were generated because we were passing /MTd-like
arguments twice, due to us and CMake 3.15+ adding these arguments:
https://cmake.org/cmake/help/v3.25/policy/CMP0091.html.
This requires CMake 3.15 on Windows.
This fixes#5940.
Previously, the socket addresses were truncated as the `msg_namelen`
field is both input **and** output: After receiving an IPv4 packet, the
socket address field would be too short for an IPv6 address.
5748: Remove the need for sprite icon images; update Icons file r=Jupeyy a=VoxelDoesCode
It felt like that it was very unoptimized to have two images for the demo buttons. Initially, I wanted to merge it into one image, but it was a much better choice to have it so that we use the Icons font file for the sprites instead. Turns out, considering how the typeface is rendered, it looks much smoother and crisper. The contrast between the squares and the symbols is much clearer alongside!
Before the change:
After the change:
Note that there is a few compromises that had to be made with the limited symbol choices. As of publishing this there is not a symbol for a crossed out keyboard, but I hope the eye works just fine for now.
## Checklist
- [x] Tested the change ingame
- [x] Provided screenshots if it is a visual change
- [ ] Tested in combination with possibly related configuration options
- [ ] Written a unit test (especially base/) or added coverage to integration test
- [ ] Considered possible null pointers and out of bounds array indexing
- [x] Changed no physics that affect existing maps
- [ ] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssanitizer--undefinedbehavioursanitizer-or-valgrinds-memcheck) (optional)
Co-authored-by: VoxelDoesCode <bluheadcat@gmail.com>
5765: Add libpng dylib to mac dmg (fixes#5764) r=Jupeyy a=def-
<!-- What is the motivation for the changes of this pull request -->
## Checklist
- [ ] Tested the change ingame
- [ ] Provided screenshots if it is a visual change
- [ ] Tested in combination with possibly related configuration options
- [ ] Written a unit test (especially base/) or added coverage to integration test
- [ ] Considered possible null pointers and out of bounds array indexing
- [ ] Changed no physics that affect existing maps
- [ ] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssanitizer--undefinedbehavioursanitizer-or-valgrinds-memcheck) (optional)
Co-authored-by: Dennis Felsing <dennis@felsin9.de>
5733: Added map_find_env tool r=def- a=sctt
a simple tool which given an envelope id returns a list of all quads associated with that envelope, screenshot attached below.
p.s.
deen told me its not a problem to commit these kind of map tools to ddnet
Co-authored-by: sctt <scottistefano91@gmail.com>
Support is incomplete for `leak_ip_address_to_all_servers` (will only
ping the first address of each server) and for the `leak_ip` setting
(which will also only ping the first address of each server).
5506: Don't register integration test servers r=heinrich5991 a=def-
Also print out more of the diff
Should help with debugging sporadic failures like https://github.com/ddnet/ddnet/runs/7058704316?check_suite_focus=true
## Checklist
- [ ] Tested the change ingame
- [ ] Provided screenshots if it is a visual change
- [ ] Tested in combination with possibly related configuration options
- [ ] Written a unit test if it works standalone, system.c especially
- [ ] Considered possible null pointers and out of bounds array indexing
- [ ] Changed no physics that affect existing maps
- [ ] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssanitizer--undefinedbehavioursanitizer-or-valgrinds-memcheck) (optional)
Co-authored-by: def <dennis@felsin9.de>
5420: Improvments for new HUD r=def- a=C0D3D3V
for testing set `#define CLIENT_VERSIONNR 16020`
Some of the changes are only available in the next version
Impressions: https://youtu.be/E770vGp4KKY
Implements a few of the recommendations from https://github.com/ddnet/ddnet/issues/5159 and Discord
- removes all player capability messages send by the server, that are now displayed in the HUD
- ~~remove `cl_ddrace_hud` and cl_show_freeze_bars~~,
- Added option `cl_freezebars_alpha_inside_freeze` setting to change the opacity if the player is inside a freeze tile (default visible 100%)
- Added option `cl_show_jumps_indicator`
- We send now the real weapon the player is holding if he is frozen
-> for that we added a new particle "snow flakes" or frozen flakes :D that are spawned over the tees body, to indicate he is frozen.
-> new extras.png asset (+ corresponding asset tab, that can potentially be used for all sorts of extras for our mod, e.g. material particles, but in the future maybe also for other mods using a similar structure like it is for entities)
- stars are no longer send to new clients, because it is duplication to the freezebar, and we decided to not allow to disable it
- also ninja stars are no longer send. They are not vanilla, and we now have a ninja bar in the new HUD.
- display target angle from 0 to 360 (looks mirrored on the x-axis to the normal trigonometry circle, but that is just how the world coordinates are)
- dummy action display is increased in size ( but only the border margin :D not the icons. so the icons are still the same size as the icons on the left side)
- allow the server to also show old HUD elements (see https://github.com/ddnet/ddnet/pull/5427 for screenshots)
fixes#5149fixes#198closes#5159fixes#2252 ~~(But not clear if we should also display the weapon, still investigating)~~
- we do no longer display a weapon inside freeze, instead we have nice freeze particles
## Checklist
- [x] Tested the change ingame
- [ ] Provided screenshots if it is a visual change
- [ ] Tested in combination with possibly related configuration options
- [ ] Written a unit test if it works standalone, system.c especially
- [x] Considered possible null pointers and out of bounds array indexing
- [x] Changed no physics that affect existing maps
- [x] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssanitizer--undefinedbehavioursanitizer-or-valgrinds-memcheck) (optional)
Co-authored-by: c0d3d3v <c0d3d3v@mag-keinen-spam.de>
4829: Added map_replace_area tool r=def- a=sctt
Motivation:
sometimes mappers need to copy a certain area from a map to another (but also to a different position of the same map).
that's very time consuming on complex maps, as tiles and quads layers have to be manually copied one by one.
map_replace_area is designed to automatically replace all the tiles and quads from a specific area to another.
`Usage: map_replace_area <from_map> <from_x> <from_y> <to_map> <to_x> <to_y> <width> <height> <output_map>`
PoW screenshots attached below.
note: for the moment map_replace_area suppose that you are working on the same map (that's what i needed), that means source and destination maps must have the same layers structure, or else an error is returned.
but i understand it might be useful to also work on totally different maps, in that case users must be able specify layers they want to consider by associating source map layers to destination map layers (by using a config file maybe).
i'm probably going to make another PR to add this enhancement in the future.
PoW:
5343: Add rcon and chat to integration test r=def- a=ChillerDragon
Add more action to the integration test script: chat messages, chat commands, rcon commands.
This unlocks more code coverage at runtime to catch asan issues.
It adds a very restrictive regex on the log format. But I am happy to maintain that in case changes to the log output happen.
The current version is already supporting heinrichs refactor https://github.com/ddnet/ddnet/pull/5036
Also ensure chat messages arrive in the correct format. Would have catched the following issues:
https://github.com/ddnet/ddnet/issues/5342https://github.com/ddnet/ddnet/issues/5340https://github.com/ddnet/ddnet/issues/5302https://github.com/ddnet/ddnet/pull/5126
DEPENDS ON:
https://github.com/ddnet/ddnet/issues/5342https://github.com/ddnet/ddnet/issues/5340
5465: Change from pnglite to libpng for PNG reading r=def- a=heinrich5991
This is desirable mainly because libpng is maintained and pnglite is
not. pnglite was last updated in 2007 (15 years ago) and probably has a
lot of security vulnerabilities.
libpng is an actively maintained library also used by browsers like
Firefox or Chromium, so it's less likely to contain security
vulnerabilities, also it's more likely to be packaged by Linux
distributions.
This also refuses to load images of types not supported by pnglite,
which allows us to think about backward compatibility while also
introducing libpng.
## Checklist
- [x] Tested the change ingame
- [ ] Provided screenshots if it is a visual change
- [ ] Tested in combination with possibly related configuration options
- [ ] Written a unit test if it works standalone, system.c especially
- [ ] Considered possible null pointers and out of bounds array indexing
- [ ] Changed no physics that affect existing maps
- [ ] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssanitizer--undefinedbehavioursanitizer-or-valgrinds-memcheck) (optional)
Co-authored-by: sctt <scottistefano91@gmail.com>
Co-authored-by: term <term@term.sinervis.pri>
Co-authored-by: f <scottistefano91@gmail.com>
Co-authored-by: heinrich5991 <heinrich5991@gmail.com>
Co-authored-by: ChillerDragon <ChillerDragon@gmail.com>
Co-authored-by: def <dennis@felsin9.de>
Co-authored-by: Jupeyy <jupjopjap@gmail.com>
This is desirable mainly because libpng is maintained and pnglite is
not. pnglite was last updated in 2007 (15 years ago) and probably has a
lot of security vulnerabilities.
libpng is an actively maintained library also used by browsers like
Firefox or Chromium, so it's less likely to contain security
vulnerabilities, also it's more likely to be packaged by Linux
distributions.
5260: Pr thread safety negative r=heinrich5991 a=def-
WorkerThread is hard because `REQUIRES(!((CJobPool *)pUser)->m_Lock)` would require alias analysis or the function using that everywhere.
https://clang.llvm.org/docs/ThreadSafetyAnalysis.html#negative
## Checklist
- [ ] Tested the change ingame
- [ ] Provided screenshots if it is a visual change
- [ ] Tested in combination with possibly related configuration options
- [ ] Written a unit test if it works standalone, system.c especially
- [ ] Considered possible null pointers and out of bounds array indexing
- [ ] Changed no physics that affect existing maps
- [ ] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssanitizer--undefinedbehavioursanitizer-or-valgrinds-memcheck) (optional)
Co-authored-by: Dennis Felsing <dennis@felsin9.de>
5264: Prevent users from setting port 1-1023 r=Jupeyy a=def-
Following user report that they set cl_port 1 and it didn't work
<!-- What is the motivation for the changes of this pull request -->
## Checklist
- [x] Tested the change ingame
- [ ] Provided screenshots if it is a visual change
- [ ] Tested in combination with possibly related configuration options
- [ ] Written a unit test if it works standalone, system.c especially
- [ ] Considered possible null pointers and out of bounds array indexing
- [ ] Changed no physics that affect existing maps
- [ ] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssanitizer--undefinedbehavioursanitizer-or-valgrinds-memcheck) (optional)
5266: Use std::make_unique (fixes#5166) r=Jupeyy a=def-
No idea why clang-tidy's modernize-... didn't work
<!-- What is the motivation for the changes of this pull request -->
## Checklist
- [ ] Tested the change ingame
- [ ] Provided screenshots if it is a visual change
- [ ] Tested in combination with possibly related configuration options
- [ ] Written a unit test if it works standalone, system.c especially
- [ ] Considered possible null pointers and out of bounds array indexing
- [ ] Changed no physics that affect existing maps
- [ ] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssanitizer--undefinedbehavioursanitizer-or-valgrinds-memcheck) (optional)
Co-authored-by: def <dennis@felsin9.de>
ld64.lld: warning: /opt/homebrew/opt/freetype/lib/libfreetype.6.dylib has version 12.0.0, which is newer than target minimum of 11.0.0
ld64.lld: error: LC_DYLD_INFO_ONLY not found in /opt/homebrew/opt/freetype/lib/libfreetype.6.dylib
ld64.lld: warning: /usr/local/lib/libGLEW.2.2.0.dylib has version 12.0.0, which is newer than target minimum of 11.0.0
ld64.lld: error: LC_DYLD_INFO_ONLY not found in /usr/local/lib/libGLEW.2.2.0.dylib
ld64.lld: warning: /opt/homebrew/Cellar/wavpack/5.4.0/lib/libwavpack.dylib has version 12.0.0, which is newer than target minimum of 11.0.0
ld64.lld: error: LC_DYLD_INFO_ONLY not found in /opt/homebrew/Cellar/wavpack/5.4.0/lib/libwavpack.dylib
ld64.lld: warning: /opt/homebrew/Cellar/ffmpeg/5.0.1/lib/libavformat.59.dylib has version 12.0.0, which is newer than target minimum of 11.0.0
ld64.lld: error: LC_DYLD_INFO_ONLY not found in /opt/homebrew/Cellar/ffmpeg/5.0.1/lib/libavformat.59.dylib
ld64.lld: warning: /opt/homebrew/Cellar/ffmpeg/5.0.1/lib/libavcodec.59.dylib has version 12.0.0, which is newer than target minimum of 11.0.0
ld64.lld: error: LC_DYLD_INFO_ONLY not found in /opt/homebrew/Cellar/ffmpeg/5.0.1/lib/libavcodec.59.dylib
ld64.lld: warning: /opt/homebrew/Cellar/ffmpeg/5.0.1/lib/libavutil.57.dylib has version 12.0.0, which is newer than target minimum of 11.0.0
ld64.lld: error: LC_DYLD_INFO_ONLY not found in /opt/homebrew/Cellar/ffmpeg/5.0.1/lib/libavutil.57.dylib
ld64.lld: warning: /opt/homebrew/Cellar/ffmpeg/5.0.1/lib/libswscale.6.dylib has version 12.0.0, which is newer than target minimum of 11.0.0
ld64.lld: error: LC_DYLD_INFO_ONLY not found in /opt/homebrew/Cellar/ffmpeg/5.0.1/lib/libswscale.6.dylib
ld64.lld: warning: /opt/homebrew/Cellar/ffmpeg/5.0.1/lib/libswresample.4.dylib has version 12.0.0, which is newer than target minimum of 11.0.0
ld64.lld: error: LC_DYLD_INFO_ONLY not found in /opt/homebrew/Cellar/ffmpeg/5.0.1/lib/libswresample.4.dylib
ld64.lld: warning: /opt/homebrew/Cellar/opusfile/0.12/lib/libopusfile.dylib has version 12.0.0, which is newer than target minimum of 11.0.0
ld64.lld: error: LC_DYLD_INFO_ONLY not found in /opt/homebrew/Cellar/opusfile/0.12/lib/libopusfile.dylib
ld64.lld: warning: /opt/homebrew/Cellar/opus/1.3.1/lib/libopus.dylib has version 12.0.0, which is newer than target minimum of 11.0.0
ld64.lld: error: LC_DYLD_INFO_ONLY not found in /opt/homebrew/Cellar/opus/1.3.1/lib/libopus.dylib
ld64.lld: warning: /opt/homebrew/Cellar/libogg/1.3.5/lib/libogg.dylib has version 12.0.0, which is newer than target minimum of 11.0.0
ld64.lld: error: LC_DYLD_INFO_ONLY not found in /opt/homebrew/Cellar/libogg/1.3.5/lib/libogg.dylib
ld64.lld: error: undefined symbol: ___glewBindSampler
>>> referenced by CMakeFiles/DDNet.dir/src/engine/client/backend/opengl/backend_opengl.cpp.o
ld64.lld: error: undefined symbol: ___glewBindSampler
>>> referenced by CMakeFiles/DDNet.dir/src/engine/client/backend/opengl/backend_opengl.cpp.o
[...]
Registering
-----------
The idea is that game servers push their server info to the
masterservers every 15 seconds or when the server info changes, but not
more than once per second.
The game servers do not support the old registering protocol anymore,
the backward compatibility is handled by the masterserver.
The register call is a HTTP POST to a URL like
`https://master1.ddnet.tw/ddnet/15/register` and looks like this:
```json
POST /ddnet/15/register HTTP/1.1
Address: tw-0.6+udp://connecting-address.invalid:8303
Secret: 81fa3955-6f83-4290-818d-31c0906b1118
Challenge-Secret: 81fa3955-6f83-4290-818d-31c0906b1118:tw0.6/ipv6
Info-Serial: 0
{
"max_clients": 64,
"max_players": 64,
"passworded": false,
"game_type": "TestDDraceNetwork",
"name": "My DDNet server",
"map": {
"name": "dm1",
"sha256": "0b0c481d77519c32fbe85624ef16ec0fa9991aec7367ad538bd280f28d8c26cf",
"size": 5805
},
"version": "0.6.4, 16.0.3",
"clients": []
}
```
The `Address` header declares that the server wants to register itself as
a `tw-0.6+udp` server, i.e. a server speaking a Teeworlds-0.6-compatible
protocol.
The free-form `Secret` header is used as a server identity, the server
list will be deduplicated via this secret.
The free-form `Challenge-Secret` is sent back via UDP for a port forward
check. This might have security implications as the masterserver can be
asked to send a UDP packet containing some user-controlled bytes. This
is somewhat mitigated by the fact that it can only go to an
attacker-controlled IP address.
The `Info-Serial` header is an integer field that should increase each
time the server info (in the body) changes. The masterserver uses that
field to ensure that it doesn't use old server infos.
The body is a free-form JSON object set by the game server. It should
contain certain keys in the correct form to be accepted by clients. The
body is optional if the masterserver already confirmed the reception of
the info with the given `Info-Serial`.
Not shown in this payload is the `Connless-Token` header that is used
for Teeworlds 0.7 style communication.
Also not shown is the `Challenge-Token` that should be included once the
server receives the challenge token via UDP.
The masterserver responds with a `200 OK` with a body like this:
```
{"status":"success"}
```
The `status` field can be `success` if the server was successfully
registered on the masterserver, `need_challenge` if the masterserver
wants the correct `Challenge-Token` header before the register process
is successful, `need_info` if the server sent an empty body but the
masterserver doesn't actually know the server info.
It can also be `error` if the request was malformed, only in this case
an HTTP status code except `200 OK` is sent.
Synchronization
---------------
The masterserver keeps state and outputs JSON files every second.
```json
{
"servers": [
{
"addresses": [
"tw-0.6+udp://127.0.0.1:8303",
"tw-0.6+udp://[::1]:8303"
],
"info_serial": 0,
"info": {
"max_clients": 64,
"max_players": 64,
"passworded": false,
"game_type": "TestDDraceNetwork",
"name": "My DDNet server",
"map": {
"name": "dm1",
"sha256": "0b0c481d77519c32fbe85624ef16ec0fa9991aec7367ad538bd280f28d8c26cf",
"size": 5805
},
"version": "0.6.4, 16.0.3",
"clients": []
}
}
]
}
```
`servers.json` (or configured by `--out`) is a server list that is
compatible with DDNet 15.5+ clients. It is a JSON object containing a
single key `servers` with a list of game servers. Each game server is
represented by a JSON object with an `addresses` key containing a list
of all known addresses of the server and an `info` key containing the
free-form server info sent by the game server. The free-form `info` JSON
object re-encoded by the master server and thus canonicalized and
stripped of any whitespace characters outside strings.
```json
{
"kind": "mastersrv",
"now": 1816002,
"secrets": {
"tw-0.6+udp://127.0.0.1:8303": {
"ping_time": 1811999,
"secret": "42d8f991-f2fa-46e5-a9ae-ebcc93846feb"
},
"tw-0.6+udp://[::1]:8303": {
"ping_time": 1811999,
"secret": "42d8f991-f2fa-46e5-a9ae-ebcc93846feb"
}
},
"servers": {
"42d8f991-f2fa-46e5-a9ae-ebcc93846feb": {
"info_serial": 0,
"info": {
"max_clients": 64,
"max_players": 64,
"passworded": false,
"game_type": "TestDDraceNetwork",
"name": "My DDNet server",
"map": {
"name": "dm1",
"sha256": "0b0c481d77519c32fbe85624ef16ec0fa9991aec7367ad538bd280f28d8c26cf",
"size": 5805
},
"version": "0.6.4, 16.0.3",
"clients": []
}
}
}
}
```
`--write-dump` outputs a JSON file compatible with `--read-dump-dir`,
this can be used to synchronize servers across different masterservers.
`--read-dump-dir` is also used to ingest servers from the backward
compatibility layer that pings each server for their server info using
the old protocol.
The `kind` field describe that this is `mastersrv` output and not from a
`backcompat`. This is used for prioritizing `mastersrv` information over
`backcompat` information.
The `now` field contains an integer describing the current time in
milliseconds relative an unspecified epoch that is fixed for each JSON
file. This is done instead of using the current time as the epoch for
better compression of non-changing data.
`secrets` is a map from each server address and to a JSON object
containing the last ping time (`ping_time`) in milliseconds relative to
the same epoch as before, and the server secret (`secret`) that is used
to unify server infos from different addresses of the same logical
server.
`servers` is a map from the aforementioned `secret`s to the
corresponding `info_serial` and `info`.
```json
[
"tw-0.6+udp://127.0.0.1:8303",
"tw-0.6+udp://[::1]:8303"
]
```
`--write-addresses` outputs a JSON file containing all addresses
corresponding to servers that are registered to HTTP masterservers. It
does not contain the servers that are obtained via backward
compatibility measures.
This file can be used by an old-style masterserver to also list
new-style servers without the game servers having to register there.
An implementation of this can be found at
https://github.com/heinrich5991/teeworlds/tree/mastersrv_6_backcompat
for Teeworlds 0.5/0.6 masterservers and at
https://github.com/heinrich5991/teeworlds/tree/mastersrv_7_backcompat
for Teeworlds 0.7 masterservers.
All these JSON files can be sent over the network in an efficient way
using https://github.com/heinrich5991/twmaster-collect. It establishes a
zstd-compressed TCP connection authenticated by a string token that is
sent in plain-text. It watches the specified file and transmits it every
time it changes. Due to the zstd-compression, the data sent over the
network is similar to the size of a diff.
Implementation
--------------
The masterserver implementation was done in Rust.
The current gameserver register implementation doesn't support more than
one masterserver for registering.
We only have C++ source files left, take advantage of this and check
whether the C++ compiler supports the flags, not the C compiler. GCC for
example warns about `-Wsuggest-override` in C mode, but supports it in
C++ mode.
5119: Use STUN to determine UDP connectivity and show diagnostics r=def- a=heinrich5991
These diagnostics are supposed to guide the user to problem resolution.
They're displayed if no packet is received from the server within one
second of connecting.
No message if we don't have STUN servers.
"Trying to determine UDP connectivity..." if no answer has been received
from the STUN server yet and it hasn't timed out yet.
"UDP seems to be filtered." if the STUN request has timed out.
"UDP and TCP IP addresses seem to be different. Try disabling VPN,
proxy or network accelerators." if the STUN request has returned an IP
address different from the one obtained via HTTP from info2.ddnet.tw.
"No answer from server yet." otherwise, if the STUN request has returned
no interesting data, indicating that it's likely the game server's
fault.
## Checklist
- [x] Tested the change ingame
- [x] Provided screenshots if it is a visual change
- [ ] Tested in combination with possibly related configuration options
- [ ] Written a unit test if it works standalone, system.c especially
- [ ] Considered possible null pointers and out of bounds array indexing
- [ ] Changed no physics that affect existing maps
- [ ] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssanitizer--undefinedbehavioursanitizer-or-valgrinds-memcheck) (optional)
Co-authored-by: heinrich5991 <heinrich5991@gmail.com>
These diagnostics are supposed to guide the user to problem resolution.
They're displayed if no packet is received from the server within one
second of connecting.
No message if we don't have STUN servers.
"Trying to determine UDP connectivity..." if no answer has been received
from the STUN server yet and it hasn't timed out yet.
"UDP seems to be filtered." if the STUN request has timed out.
"UDP and TCP IP addresses seem to be different. Try disabling VPN,
proxy or network accelerators." if the STUN request has returned an IP
address different from the one obtained via HTTP from info2.ddnet.tw.
"No answer from server yet." otherwise, if the STUN request has returned
no interesting data, indicating that it's likely the game server's
fault.
ChillerDragon
