Support adding up to three communities as favorites in the server browser. Favorites can be changed with favorite buttons which are shown in the community filter on the Internet and Favorites tabs. The commands `add_favorite_community` and `remove_favorite_community` are added to change the favorite communities via the console and for saving the favorite communities to the config file. For the favorite communities, additional tabs using the communities' icons are shown in the server browser next to the Internet, LAN and Favorites tabs. Each community tab shows only the servers from the respective community, hence the community filters UI is not shown on the community tabs but only on the Internet and Favorites tabs. The country and type filters on community tabs cover only the countries and types from the respective community. Favorite communities are added from left to right. When more than three favorite communities are added, the oldest (leftmost) favorite community will be removed from the list.
When starting the client for the first time, i.e. with `cl_show_welcome 1`, the DDNet tab will be created as the only favorite community and selected initially. The community, country and type filters are unset when starting for the first time, so the Internet tab now shows all servers per default.
When starting with a `ui_page` for a favorite community that is not configured, the page is reset to the Internet tab. This also affects those who upgrade from versions with the old DDNet and KoG tabs. The server browser is now also correctly updated when changing `ui_page` via the console.
Track country and type filters for every community separately, to avoid filters resetting when switching between community tabs or changing the community filter. The commands `add_excluded_community`, `remove_excluded_community`, `add_excluded_country`, `remove_excluded_country`, `add_excluded_type` and `remove_excluded_type` are added to change the exclusion filters via the console and for saving the exclusion filters to the config file.
Render community filters above the toolbox (filter, info and friends) tabs when on the Internet and Favorites tab, so this setting is more visible and can be changed also when the other toolbox tabs are selected.
Add icon for the none community, based on the tee country flag color. This icon is hard-coded in the client, as the none community also is, so fetching the icon from the server would be inconvenient. Load community icons already when rendering the menu instead of only when rendering the server browser, so the icons are immediately available when using the start menu.
Find tutorial server by searching for community type "Tutorial" instead of searching for "(Tutorial)" in the server name.
Avoid cleaning favorite communities and filters when there are no communities, i.e. when the DDNet info failed to be loaded or does not contain any communities, to avoid losing all favorite communities and filters in this case.
Closes#7774.
Remove the DDNet and KoG tabs and replace them with a community filter, which works like the country and type filters. Community information (ID, name, icon SHA256, countries, types, servers, ranked maps) is loaded dynamically from the DDNet info json.
The official checkmark icon is replaced with community icons. The icons are downloaded to the `communityicons` folder in the config directory from a URL specified in the DDNet info json. The icons are only re-downloaded when their SHA256 has changed.
Per default, the Internet tab with all communities is selected. When starting the client for the first time, only the DDNet community is selected. The community, country and type filters can also be used on the Favorites tab now. The LAN tab remains unchanged.
The country and type filters are refactored. Previously, using these filters changed the total number of servers and players that were displayed, as the filter was implemented as a separate step. Now the community, country and type filters work like the other browser filters. This also has the benefit that the server list and DDNet info json are not reloaded each time these filters are changed, as updating the local filtering/sorting is enough now.
The check for finished maps is made more efficient by using an `std::unordered_set` instead of linear search.
Closes#5654.
Use config manager to reset `ui_page` to the default, instead of using `CMenus::PAGE_DDNET` directly (which is the default).
Check current type of serverbrowser instead of checking the current `ui_page` against `CMenus::PAGE_DDNET` and `CMenus::PAGE_KOG`.
Replace existing hard-coded support for two "networks" with support for a dynamic list of "communities" which are indexed by a string-ID, though right now the DDNet and KoG communities are still hard-coded. The communities now also support an arbitrary number of associated countries, types and servers. This is a refactoring to prepare for getting the list of communities dynamically from the server and removing the DDNet and KoG tabs from the serverbrowser.
The variable `m_ResortServerBrowser` was only used in the client to pass it to the server browser. For better separation of concerns, this variable is moved inside `CServerBrowser` and the function `RequestResort` should be called to set it to `true`.
The existing variable `m_SortOnNextUpdate` is replaced with this, as it served the same purpose already. The variable name from upstream is used to reduce conflicts.
It was only being used for "Leak IP" favorites and LAN servers. Since
KoG has upgraded to latest DDNet, there aren't much servers left that
don't understand the new 64 player info protocol.
Keywords: fstd, dtsf
Support is incomplete for `leak_ip_address_to_all_servers` (will only
ping the first address of each server) and for the `leak_ip` setting
(which will also only ping the first address of each server).
Fix pointer and pointer array variable naming
Huge renaming to match our rules
Used regex: (?!(return|delete)\b)\b\w+ (m_|ms_|g_|gs_|s_)[^a]\w+\[
(?!(return|delete)\b)\b\w+ (?!(m_|ms_|g_|gs_|s_))[^a]\w+\[
Further format static variables
Format almost all pointer names accordingly
Used regex: (?!(return)\b)\b\w+
\*(?!(m_p|p|s_p|m_ap|s_ap|g_p|g_ap|ap|gs_ap|ms_ap|gs_p|ms_p))\w+\b[^:\(p]
clang-format
Fix CI fail
Fix misnamed non pointer as pointer and non array as array
Used regex: (?!(return|delete)\b)\b\w+ (m_|ms_|g_|gs_|s_)p\w+\b
(?!return\b)\b\w+ (ms_|m_|g_|gs_|s_)a\w+\b[^\[]
clang-format
Revert to SCREAMING_SNAKE_CASE and reinstate dead code
Registering
-----------
The idea is that game servers push their server info to the
masterservers every 15 seconds or when the server info changes, but not
more than once per second.
The game servers do not support the old registering protocol anymore,
the backward compatibility is handled by the masterserver.
The register call is a HTTP POST to a URL like
`https://master1.ddnet.tw/ddnet/15/register` and looks like this:
```json
POST /ddnet/15/register HTTP/1.1
Address: tw-0.6+udp://connecting-address.invalid:8303
Secret: 81fa3955-6f83-4290-818d-31c0906b1118
Challenge-Secret: 81fa3955-6f83-4290-818d-31c0906b1118:tw0.6/ipv6
Info-Serial: 0
{
"max_clients": 64,
"max_players": 64,
"passworded": false,
"game_type": "TestDDraceNetwork",
"name": "My DDNet server",
"map": {
"name": "dm1",
"sha256": "0b0c481d77519c32fbe85624ef16ec0fa9991aec7367ad538bd280f28d8c26cf",
"size": 5805
},
"version": "0.6.4, 16.0.3",
"clients": []
}
```
The `Address` header declares that the server wants to register itself as
a `tw-0.6+udp` server, i.e. a server speaking a Teeworlds-0.6-compatible
protocol.
The free-form `Secret` header is used as a server identity, the server
list will be deduplicated via this secret.
The free-form `Challenge-Secret` is sent back via UDP for a port forward
check. This might have security implications as the masterserver can be
asked to send a UDP packet containing some user-controlled bytes. This
is somewhat mitigated by the fact that it can only go to an
attacker-controlled IP address.
The `Info-Serial` header is an integer field that should increase each
time the server info (in the body) changes. The masterserver uses that
field to ensure that it doesn't use old server infos.
The body is a free-form JSON object set by the game server. It should
contain certain keys in the correct form to be accepted by clients. The
body is optional if the masterserver already confirmed the reception of
the info with the given `Info-Serial`.
Not shown in this payload is the `Connless-Token` header that is used
for Teeworlds 0.7 style communication.
Also not shown is the `Challenge-Token` that should be included once the
server receives the challenge token via UDP.
The masterserver responds with a `200 OK` with a body like this:
```
{"status":"success"}
```
The `status` field can be `success` if the server was successfully
registered on the masterserver, `need_challenge` if the masterserver
wants the correct `Challenge-Token` header before the register process
is successful, `need_info` if the server sent an empty body but the
masterserver doesn't actually know the server info.
It can also be `error` if the request was malformed, only in this case
an HTTP status code except `200 OK` is sent.
Synchronization
---------------
The masterserver keeps state and outputs JSON files every second.
```json
{
"servers": [
{
"addresses": [
"tw-0.6+udp://127.0.0.1:8303",
"tw-0.6+udp://[::1]:8303"
],
"info_serial": 0,
"info": {
"max_clients": 64,
"max_players": 64,
"passworded": false,
"game_type": "TestDDraceNetwork",
"name": "My DDNet server",
"map": {
"name": "dm1",
"sha256": "0b0c481d77519c32fbe85624ef16ec0fa9991aec7367ad538bd280f28d8c26cf",
"size": 5805
},
"version": "0.6.4, 16.0.3",
"clients": []
}
}
]
}
```
`servers.json` (or configured by `--out`) is a server list that is
compatible with DDNet 15.5+ clients. It is a JSON object containing a
single key `servers` with a list of game servers. Each game server is
represented by a JSON object with an `addresses` key containing a list
of all known addresses of the server and an `info` key containing the
free-form server info sent by the game server. The free-form `info` JSON
object re-encoded by the master server and thus canonicalized and
stripped of any whitespace characters outside strings.
```json
{
"kind": "mastersrv",
"now": 1816002,
"secrets": {
"tw-0.6+udp://127.0.0.1:8303": {
"ping_time": 1811999,
"secret": "42d8f991-f2fa-46e5-a9ae-ebcc93846feb"
},
"tw-0.6+udp://[::1]:8303": {
"ping_time": 1811999,
"secret": "42d8f991-f2fa-46e5-a9ae-ebcc93846feb"
}
},
"servers": {
"42d8f991-f2fa-46e5-a9ae-ebcc93846feb": {
"info_serial": 0,
"info": {
"max_clients": 64,
"max_players": 64,
"passworded": false,
"game_type": "TestDDraceNetwork",
"name": "My DDNet server",
"map": {
"name": "dm1",
"sha256": "0b0c481d77519c32fbe85624ef16ec0fa9991aec7367ad538bd280f28d8c26cf",
"size": 5805
},
"version": "0.6.4, 16.0.3",
"clients": []
}
}
}
}
```
`--write-dump` outputs a JSON file compatible with `--read-dump-dir`,
this can be used to synchronize servers across different masterservers.
`--read-dump-dir` is also used to ingest servers from the backward
compatibility layer that pings each server for their server info using
the old protocol.
The `kind` field describe that this is `mastersrv` output and not from a
`backcompat`. This is used for prioritizing `mastersrv` information over
`backcompat` information.
The `now` field contains an integer describing the current time in
milliseconds relative an unspecified epoch that is fixed for each JSON
file. This is done instead of using the current time as the epoch for
better compression of non-changing data.
`secrets` is a map from each server address and to a JSON object
containing the last ping time (`ping_time`) in milliseconds relative to
the same epoch as before, and the server secret (`secret`) that is used
to unify server infos from different addresses of the same logical
server.
`servers` is a map from the aforementioned `secret`s to the
corresponding `info_serial` and `info`.
```json
[
"tw-0.6+udp://127.0.0.1:8303",
"tw-0.6+udp://[::1]:8303"
]
```
`--write-addresses` outputs a JSON file containing all addresses
corresponding to servers that are registered to HTTP masterservers. It
does not contain the servers that are obtained via backward
compatibility measures.
This file can be used by an old-style masterserver to also list
new-style servers without the game servers having to register there.
An implementation of this can be found at
https://github.com/heinrich5991/teeworlds/tree/mastersrv_6_backcompat
for Teeworlds 0.5/0.6 masterservers and at
https://github.com/heinrich5991/teeworlds/tree/mastersrv_7_backcompat
for Teeworlds 0.7 masterservers.
All these JSON files can be sent over the network in an efficient way
using https://github.com/heinrich5991/twmaster-collect. It establishes a
zstd-compressed TCP connection authenticated by a string token that is
sent in plain-text. It watches the specified file and transmits it every
time it changes. Due to the zstd-compression, the data sent over the
network is similar to the size of a diff.
Implementation
--------------
The masterserver implementation was done in Rust.
The current gameserver register implementation doesn't support more than
one masterserver for registering.
Thanks to @Jupeyy for figuring out this bug.
Previously, the client would send the request for server info and the
server info ping packet with the same token, making the answer
indistinguishable from each other. Fix this by generating a token that
won't collide.
Fixes#3862.
Automatically determine the approximate location of the client by
including that info in the already fetched https://info2.ddnet.tw/info.
Can be overridden using the the config variable `br_location`.
Summary
=======
The idea of this is that clients will not have to ping each server for
server infos which takes long, leaks the client's IP address even to
servers the user does not join and is a DoS vector of the game servers
for attackers.
For the Internet, DDNet and KoG tab, the server list is entirely fetched
from the master server, filtering out servers that don't belong into the
list.
The favorites tab is also supposed to work that way, except for servers
that are marked as "also ping this server if it's not in the master
server list".
The LAN tab continues to broadcast the server info packet to find
servers in the LAN.
How does it work?
=================
The client ships with a list of master server list URLs. On first start,
the client checks which of these work and selects the fastest one.
Querying the server list is a HTTP GET request on that URL. The
response is a JSON document that contains server infos, server addresses
as URLs and an approximate location.
It can also contain a legacy server list which is a list of bare IP
addresses similar to the functionality the old master servers provided
via UDP. This allows us to backtrack on the larger update if it won't
work out.
Lost functionality
==================
(also known as user-visible changes)
Since the client doesn't ping each server in the list anymore, it has no
way of knowing its latency to the servers.
This is alleviated a bit by providing an approximate location for each
server (continent) so the client only has to know its own location for
approximating pings.
Not planning to do this automatically, but at least cleaning it up once
provides some benefit. Every header should include what it uses.
$ for i in src/**/*.h; do j=${i#"src/"}; echo $j; echo "#include <$j>\nint main() { return 0; }" | /usr/bin/c++ -DCONF_OPENSSL -DCONF_SQL -DCONF_VIDEORECORDER -DCONF_WAVPACK_CLOSE_FILE -DCONF_WAVPACK_OPEN_FILE_INPUT_EX -DGAME_RELEASE_VERSION=\"15.0.5\" -DGLEW_STATIC -D_FORTIFY_SOURCE=2 -I/usr/include/freetype2 -I/usr/include/opus -I/usr/include/SDL2 -I/usr/include/wavpack -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/include/libmount -I/usr/include/blkid -Isrc -I/usr/include/mysql -I/home/deen/sys/include/ -O2 -g -DNDEBUG -fdiagnostics-color=always -fstack-protector-all -Wall -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wformat=2 -Wno-nullability-completeness -Wduplicated-cond -Wduplicated-branches -Wlogical-op -Wrestrict -std=gnu++11 -o /dev/null -x c++ -; done
Ignored: tuning.h, variables.h, config_common.h, mapitems_ex_types.h, mapbugs_list.h, protocol7.h, teehistorian_ex_chunks.h, protocol_ex_msgs.h, config.h, config_variables.h, external, keynames.h
Purely automatic change. In case of conflict with this change, apply the
other change and rerun the formatting to restore it:
$ python scripts/fix_style.py
- As requested by qshar and KoG players
- Similar to DDNet tab
- Info fetched from servers-kog entry from https://info.ddnet.tw/info
- Also supports countries and types
- Doesn't inform whether map has been finished
- Generalized the code a bit but it's still ugly
- Depends on #1533, also shows KoG servers as official/verified
