Commit graph

47 commits

Author SHA1 Message Date
Chairn b80af9beb3 Fix wrongly corrected naming introduced in #5391 2022-06-15 21:09:57 +02:00
Chairn 01edaec628 Format vector variables names (fixes #5209) 2022-06-11 21:38:49 +02:00
heinrich5991 6b65ccb945 Add HTTP masterserver registering and HTTP masterserver
Registering
-----------

The idea is that game servers push their server info to the
masterservers every 15 seconds or when the server info changes, but not
more than once per second.

The game servers do not support the old registering protocol anymore,
the backward compatibility is handled by the masterserver.

The register call is a HTTP POST to a URL like
`https://master1.ddnet.tw/ddnet/15/register` and looks like this:
```json
POST /ddnet/15/register HTTP/1.1
Address: tw-0.6+udp://connecting-address.invalid:8303
Secret: 81fa3955-6f83-4290-818d-31c0906b1118
Challenge-Secret: 81fa3955-6f83-4290-818d-31c0906b1118:tw0.6/ipv6
Info-Serial: 0

{
	"max_clients": 64,
	"max_players": 64,
	"passworded": false,
	"game_type": "TestDDraceNetwork",
	"name": "My DDNet server",
	"map": {
		"name": "dm1",
		"sha256": "0b0c481d77519c32fbe85624ef16ec0fa9991aec7367ad538bd280f28d8c26cf",
		"size": 5805
	},
	"version": "0.6.4, 16.0.3",
	"clients": []
}
```

The `Address` header declares that the server wants to register itself as
a `tw-0.6+udp` server, i.e. a server speaking a Teeworlds-0.6-compatible
protocol.

The free-form `Secret` header is used as a server identity, the server
list will be deduplicated via this secret.

The free-form `Challenge-Secret` is sent back via UDP for a port forward
check.  This might have security implications as the masterserver can be
asked to send a UDP packet containing some user-controlled bytes. This
is somewhat mitigated by the fact that it can only go to an
attacker-controlled IP address.

The `Info-Serial` header is an integer field that should increase each
time the server info (in the body) changes. The masterserver uses that
field to ensure that it doesn't use old server infos.

The body is a free-form JSON object set by the game server. It should
contain certain keys in the correct form to be accepted by clients. The
body is optional if the masterserver already confirmed the reception of
the info with the given `Info-Serial`.

Not shown in this payload is the `Connless-Token` header that is used
for Teeworlds 0.7 style communication.

Also not shown is the `Challenge-Token` that should be included once the
server receives the challenge token via UDP.

The masterserver responds with a `200 OK` with a body like this:

```
{"status":"success"}
```

The `status` field can be `success` if the server was successfully
registered on the masterserver, `need_challenge` if the masterserver
wants the correct `Challenge-Token` header before the register process
is successful, `need_info` if the server sent an empty body but the
masterserver doesn't actually know the server info.

It can also be `error` if the request was malformed, only in this case
an HTTP status code except `200 OK` is sent.

Synchronization
---------------

The masterserver keeps state and outputs JSON files every second.

```json
{
	"servers": [
		{
			"addresses": [
				"tw-0.6+udp://127.0.0.1:8303",
				"tw-0.6+udp://[::1]:8303"
			],
			"info_serial": 0,
			"info": {
				"max_clients": 64,
				"max_players": 64,
				"passworded": false,
				"game_type": "TestDDraceNetwork",
				"name": "My DDNet server",
				"map": {
					"name": "dm1",
					"sha256": "0b0c481d77519c32fbe85624ef16ec0fa9991aec7367ad538bd280f28d8c26cf",
					"size": 5805
				},
				"version": "0.6.4, 16.0.3",
				"clients": []
			}
		}
	]
}
```

`servers.json` (or configured by `--out`) is a server list that is
compatible with DDNet 15.5+ clients. It is a JSON object containing a
single key `servers` with a list of game servers. Each game server is
represented by a JSON object with an `addresses` key containing a list
of all known addresses of the server and an `info` key containing the
free-form server info sent by the game server. The free-form `info` JSON
object re-encoded by the master server and thus canonicalized and
stripped of any whitespace characters outside strings.

```json
{
	"kind": "mastersrv",
	"now": 1816002,
	"secrets": {
		"tw-0.6+udp://127.0.0.1:8303": {
			"ping_time": 1811999,
			"secret": "42d8f991-f2fa-46e5-a9ae-ebcc93846feb"
		},
		"tw-0.6+udp://[::1]:8303": {
			"ping_time": 1811999,
			"secret": "42d8f991-f2fa-46e5-a9ae-ebcc93846feb"
		}
	},
	"servers": {
		"42d8f991-f2fa-46e5-a9ae-ebcc93846feb": {
			"info_serial": 0,
			"info": {
				"max_clients": 64,
				"max_players": 64,
				"passworded": false,
				"game_type": "TestDDraceNetwork",
				"name": "My DDNet server",
				"map": {
					"name": "dm1",
					"sha256": "0b0c481d77519c32fbe85624ef16ec0fa9991aec7367ad538bd280f28d8c26cf",
					"size": 5805
				},
				"version": "0.6.4, 16.0.3",
				"clients": []
			}
		}
	}
}
```

`--write-dump` outputs a JSON file compatible with `--read-dump-dir`,
this can be used to synchronize servers across different masterservers.
`--read-dump-dir` is also used to ingest servers from the backward
compatibility layer that pings each server for their server info using
the old protocol.

The `kind` field describe that this is `mastersrv` output and not from a
`backcompat`. This is used for prioritizing `mastersrv` information over
`backcompat` information.

The `now` field contains an integer describing the current time in
milliseconds relative an unspecified epoch that is fixed for each JSON
file. This is done instead of using the current time as the epoch for
better compression of non-changing data.

`secrets` is a map from each server address and to a JSON object
containing the last ping time (`ping_time`) in milliseconds relative to
the same epoch as before, and the server secret (`secret`) that is used
to unify server infos from different addresses of the same logical
server.

`servers` is a map from the aforementioned `secret`s to the
corresponding `info_serial` and `info`.

```json
[
	"tw-0.6+udp://127.0.0.1:8303",
	"tw-0.6+udp://[::1]:8303"
]
```

`--write-addresses` outputs a JSON file containing all addresses
corresponding to servers that are registered to HTTP masterservers. It
does not contain the servers that are obtained via backward
compatibility measures.

This file can be used by an old-style masterserver to also list
new-style servers without the game servers having to register there.

An implementation of this can be found at
https://github.com/heinrich5991/teeworlds/tree/mastersrv_6_backcompat
for Teeworlds 0.5/0.6 masterservers and at
https://github.com/heinrich5991/teeworlds/tree/mastersrv_7_backcompat
for Teeworlds 0.7 masterservers.

All these JSON files can be sent over the network in an efficient way
using https://github.com/heinrich5991/twmaster-collect. It establishes a
zstd-compressed TCP connection authenticated by a string token that is
sent in plain-text. It watches the specified file and transmits it every
time it changes. Due to the zstd-compression, the data sent over the
network is similar to the size of a diff.

Implementation
--------------

The masterserver implementation was done in Rust.

The current gameserver register implementation doesn't support more than
one masterserver for registering.
2022-05-20 08:58:32 +02:00
Chairn 583d6e6c01 Mark virtual function as override using a script:
while IFS= read -r line; do file=${line%%:*}; lineno=$(echo $line | cut
-d':' -f2); echo "Treating $file line $lineno"; sed -i -e
"${lineno}s/virtual //" -e "${lineno}s/);\$/) override;/" -e
"${lineno}s/)\$/) override/" -e "${lineno}s/const\$/const override/" -e
"${lineno}s/) {/) override {/" -e "${lineno}s/) const {/) const override
{/" -e "${lineno}s/const;$/const override;/" "$file"; done < a
2022-05-17 23:47:32 +02:00
heinrich5991 1b233ab95c Add the HTTP client methods to the server 2022-05-13 09:04:39 +02:00
def 3d47274f28 Add tutorial joining code and tutorial map 2022-03-19 23:35:27 +01:00
Robert Müller 74f87ee604 Remove unnecessary trailing semicolons after method declarations 2022-03-08 20:01:26 +01:00
def 653c7715a5 Add readability-avoid-const-params-in-decls 2022-01-22 13:44:22 +01:00
Jupeyy c06edd9aeb Update server list silently 2021-09-13 16:50:40 +02:00
Jupeyy a663799188 uint64 -> uint64_t, int64 -> int64_t 2021-06-24 17:19:17 +02:00
heinrich5991 97a8fcca04 Send server info request for ping with different token
Thanks to @Jupeyy for figuring out this bug.

Previously, the client would send the request for server info and the
server info ping packet with the same token, making the answer
indistinguishable from each other. Fix this by generating a token that
won't collide.

Fixes #3862.
2021-06-09 17:29:06 +02:00
heinrich5991 26a577b891 Fix crash in serverbrowser initialization
Another missing commit.

Fixes #3875.

CC #3772 #3855
2021-06-03 00:27:00 +02:00
heinrich5991 53e9457965 Save previously determined best serverlist
Also make the whole process more robust against failures, retry finding
the best serverlist if the current one is broken.
2021-05-14 01:01:09 +02:00
heinrich5991 9002c9f9a2 Allow pinging favorites directly, leaking your IP address
This is on by default for favorites added in the LAN tab, otherwise off.
2021-05-14 01:01:09 +02:00
heinrich5991 ee42faac1f Determine one's own location automatically, but provide override
Automatically determine the approximate location of the client by
including that info in the already fetched https://info2.ddnet.tw/info.
Can be overridden using the the config variable `br_location`.
2021-05-14 01:01:08 +02:00
heinrich5991 a0df2bdf9c Measure the ping to the current server occasionally
This is done using the serverbrowser extended GETINFO packet. It's done
when the client enters the game and every ten minutes afterwards.
2021-05-14 01:01:08 +02:00
heinrich5991 ed561c3602 Add a command to ping all servers
Name it in a way to make clear what information it leaks to anyone who
wants to listen.
2021-05-14 01:01:07 +02:00
heinrich5991 bce7a4f84e Add serverbrowser ping cache 2021-05-14 01:00:41 +02:00
heinrich5991 436e202cbd Add client-side HTTP server info
Summary
=======

The idea of this is that clients will not have to ping each server for
server infos which takes long, leaks the client's IP address even to
servers the user does not join and is a DoS vector of the game servers
for attackers.

For the Internet, DDNet and KoG tab, the server list is entirely fetched
from the master server, filtering out servers that don't belong into the
list.

The favorites tab is also supposed to work that way, except for servers
that are marked as "also ping this server if it's not in the master
server list".

The LAN tab continues to broadcast the server info packet to find
servers in the LAN.

How does it work?
=================

The client ships with a list of master server list URLs. On first start,
the client checks which of these work and selects the fastest one.
Querying the server list is a HTTP GET request on that URL. The
response is a JSON document that contains server infos, server addresses
as URLs and an approximate location.

It can also contain a legacy server list which is a list of bare IP
addresses similar to the functionality the old master servers provided
via UDP. This allows us to backtrack on the larger update if it won't
work out.

Lost functionality
==================

(also known as user-visible changes)

Since the client doesn't ping each server in the list anymore, it has no
way of knowing its latency to the servers.

This is alleviated a bit by providing an approximate location for each
server (continent) so the client only has to know its own location for
approximating pings.
2021-05-14 01:00:39 +02:00
Alexander Akulich c2f276cee1 Port CConfig API from the upstream (0.7.5)
The old (g_Config) API is kept to not break the stuff.

See commits:
    de5859b371
    78076761eb
2021-01-10 17:10:19 +03:00
Jupeyy fb74bcbb55 Don't sort on entry adding 2020-12-22 11:30:20 +01:00
def 7dc33abe23 Add back Australia... 2020-10-11 01:00:56 +02:00
def b178c7c74a Make sure headers compile standalone
Not planning to do this automatically, but at least cleaning it up once
provides some benefit. Every header should include what it uses.

$ for i in src/**/*.h; do j=${i#"src/"}; echo $j; echo "#include <$j>\nint main() { return 0; }" | /usr/bin/c++ -DCONF_OPENSSL -DCONF_SQL -DCONF_VIDEORECORDER -DCONF_WAVPACK_CLOSE_FILE -DCONF_WAVPACK_OPEN_FILE_INPUT_EX -DGAME_RELEASE_VERSION=\"15.0.5\" -DGLEW_STATIC -D_FORTIFY_SOURCE=2 -I/usr/include/freetype2 -I/usr/include/opus -I/usr/include/SDL2 -I/usr/include/wavpack -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/include/libmount -I/usr/include/blkid -Isrc -I/usr/include/mysql -I/home/deen/sys/include/ -O2 -g -DNDEBUG -fdiagnostics-color=always -fstack-protector-all -Wall -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wformat=2 -Wno-nullability-completeness -Wduplicated-cond -Wduplicated-branches -Wlogical-op -Wrestrict -std=gnu++11 -o /dev/null -x c++ -; done

Ignored: tuning.h, variables.h, config_common.h, mapitems_ex_types.h, mapbugs_list.h, protocol7.h, teehistorian_ex_chunks.h, protocol_ex_msgs.h, config.h, config_variables.h, external, keynames.h
2020-09-26 21:50:27 +02:00
def 3be8a592e5 Run clang-format
Purely automatic change. In case of conflict with this change, apply the
other change and rerun the formatting to restore it:

$ python scripts/fix_style.py
2020-09-26 21:50:15 +02:00
trml 4abe8caab8 Sort servers by a combination of players and ping as default 2020-08-24 23:25:45 +02:00
def bf576723a3 Add KoG tab in server browser
- As requested by qshar and KoG players
- Similar to DDNet tab
- Info fetched from servers-kog entry from https://info.ddnet.tw/info
- Also supports countries and types
- Doesn't inform whether map has been finished
- Generalized the code a bit but it's still ugly
- Depends on #1533, also shows KoG servers as official/verified
2019-03-24 23:15:40 +01:00
def ed5b08e7dc Also recheck official server status when json gets loaded 2019-03-19 07:57:09 +01:00
def faf388b853 Fix #1350 2018-10-29 22:03:57 +01:00
Dennis Felsing e2eb4a66dc Add Filter for connecting players (fixes #1263) 2018-08-21 09:03:48 +02:00
heinrich5991 16694a0d7b Use a random token for serverbrowse requests
This alleviates the risk for spoofed servers.
2017-10-14 21:58:23 +02:00
def 89ed22e8d8 Introduce ddnet-info.json
- Single json file containing all information for the client
- Fetched from https://info.ddnet.tw/info?name=deen
- Replaces versionsrv, news, ddnet-maps.json and ddnet-ranks.json
- Servers are sorted by most popular ones for respective player
- Always stays < 100 ms response time, compared to occasional 50 s for
  old ddnet-ranks.json
2017-09-03 17:42:04 +02:00
def 16f3ddc5a7 Show if user has rank on map in browser (+filter)
UI implementation by timakro
2017-08-30 21:34:04 +02:00
heinrich5991 1d81d56850 Introduce new, vanilla-compatible server info protocol
This means that we have a reliable and fast way to query for extended info,
while also not wasting network bandwidth.

The protocol is designed to be extensible, there's four bytes space for
encoding more request types (currently zeroed), and there's one string in each
response packet and one string for each player available (currently the empty
string).

The protocol itself has no problems with more than 64 players, although the
current client implementation will drop the player info after the 64th player,
because it uses a static array for storage.

Also fixes #130, the player list is just sorted each time new player info
arrives.
2017-03-29 12:56:13 +02:00
def 6e8ada1ac8 Fix whitespaces a bit 2015-07-09 02:12:35 +02:00
def b6008f5b20 Only refresh serverbrowser when necessary 2015-05-11 21:51:06 +02:00
def 8d945fe968 Exclude DDNet servery by type 2014-12-14 16:45:18 +01:00
east 202d5d8866 Filter flag grid added 2014-09-19 23:52:09 +02:00
east 9db12d455d working on CDDNetCountry 2014-09-18 16:13:06 +02:00
east e732008157 DDNet serverlist added 2014-09-13 16:36:25 +02:00
def 9e4114f27d Better 64 player search 2014-01-14 21:40:55 +01:00
def 8bcd622412 Fix stupid merge 2014-01-08 07:50:58 +01:00
Patrick Rathje 4c080fe137 Better Refreshing
Conflicts:
	src/engine/client/client.cpp
	src/engine/client/serverbrowser.cpp
	src/engine/client/serverbrowser.h
	src/engine/masterserver.h
	src/engine/shared/masterserver.cpp
2014-01-08 07:46:09 +01:00
Patrick Rathje 020bb16f54 Faster Refreshing
Conflicts:
	src/engine/client/serverbrowser.cpp
2014-01-08 07:42:19 +01:00
def 52e83313c8 64 players in server browse 2014-01-08 06:15:56 +01:00
def 280368d46a 2048 favorites 2013-10-07 03:56:39 +02:00
oy 06115dd49d added "Whitespace and line Endings cleanup" by GreYFoX 2011-04-13 20:37:12 +02:00
oy a583633cde added renaming of srvbrowse so it matches with the interface filename by Choupom 2011-03-27 18:00:54 +02:00
Renamed from src/engine/client/srvbrowse.h (Browse further)