Commit graph

78 commits

Author SHA1 Message Date
Robert Müller 4048bbbf5c Organize engine-shared includes 2022-06-17 17:39:24 +02:00
ChillerDragon 3667061b68 Use nullptr in generated and editor
-Wzero-as-null-pointer-constant
2022-06-13 18:28:13 +02:00
heinrich5991 6b65ccb945 Add HTTP masterserver registering and HTTP masterserver
Registering
-----------

The idea is that game servers push their server info to the
masterservers every 15 seconds or when the server info changes, but not
more than once per second.

The game servers do not support the old registering protocol anymore,
the backward compatibility is handled by the masterserver.

The register call is a HTTP POST to a URL like
`https://master1.ddnet.tw/ddnet/15/register` and looks like this:
```json
POST /ddnet/15/register HTTP/1.1
Address: tw-0.6+udp://connecting-address.invalid:8303
Secret: 81fa3955-6f83-4290-818d-31c0906b1118
Challenge-Secret: 81fa3955-6f83-4290-818d-31c0906b1118:tw0.6/ipv6
Info-Serial: 0

{
	"max_clients": 64,
	"max_players": 64,
	"passworded": false,
	"game_type": "TestDDraceNetwork",
	"name": "My DDNet server",
	"map": {
		"name": "dm1",
		"sha256": "0b0c481d77519c32fbe85624ef16ec0fa9991aec7367ad538bd280f28d8c26cf",
		"size": 5805
	},
	"version": "0.6.4, 16.0.3",
	"clients": []
}
```

The `Address` header declares that the server wants to register itself as
a `tw-0.6+udp` server, i.e. a server speaking a Teeworlds-0.6-compatible
protocol.

The free-form `Secret` header is used as a server identity, the server
list will be deduplicated via this secret.

The free-form `Challenge-Secret` is sent back via UDP for a port forward
check.  This might have security implications as the masterserver can be
asked to send a UDP packet containing some user-controlled bytes. This
is somewhat mitigated by the fact that it can only go to an
attacker-controlled IP address.

The `Info-Serial` header is an integer field that should increase each
time the server info (in the body) changes. The masterserver uses that
field to ensure that it doesn't use old server infos.

The body is a free-form JSON object set by the game server. It should
contain certain keys in the correct form to be accepted by clients. The
body is optional if the masterserver already confirmed the reception of
the info with the given `Info-Serial`.

Not shown in this payload is the `Connless-Token` header that is used
for Teeworlds 0.7 style communication.

Also not shown is the `Challenge-Token` that should be included once the
server receives the challenge token via UDP.

The masterserver responds with a `200 OK` with a body like this:

```
{"status":"success"}
```

The `status` field can be `success` if the server was successfully
registered on the masterserver, `need_challenge` if the masterserver
wants the correct `Challenge-Token` header before the register process
is successful, `need_info` if the server sent an empty body but the
masterserver doesn't actually know the server info.

It can also be `error` if the request was malformed, only in this case
an HTTP status code except `200 OK` is sent.

Synchronization
---------------

The masterserver keeps state and outputs JSON files every second.

```json
{
	"servers": [
		{
			"addresses": [
				"tw-0.6+udp://127.0.0.1:8303",
				"tw-0.6+udp://[::1]:8303"
			],
			"info_serial": 0,
			"info": {
				"max_clients": 64,
				"max_players": 64,
				"passworded": false,
				"game_type": "TestDDraceNetwork",
				"name": "My DDNet server",
				"map": {
					"name": "dm1",
					"sha256": "0b0c481d77519c32fbe85624ef16ec0fa9991aec7367ad538bd280f28d8c26cf",
					"size": 5805
				},
				"version": "0.6.4, 16.0.3",
				"clients": []
			}
		}
	]
}
```

`servers.json` (or configured by `--out`) is a server list that is
compatible with DDNet 15.5+ clients. It is a JSON object containing a
single key `servers` with a list of game servers. Each game server is
represented by a JSON object with an `addresses` key containing a list
of all known addresses of the server and an `info` key containing the
free-form server info sent by the game server. The free-form `info` JSON
object re-encoded by the master server and thus canonicalized and
stripped of any whitespace characters outside strings.

```json
{
	"kind": "mastersrv",
	"now": 1816002,
	"secrets": {
		"tw-0.6+udp://127.0.0.1:8303": {
			"ping_time": 1811999,
			"secret": "42d8f991-f2fa-46e5-a9ae-ebcc93846feb"
		},
		"tw-0.6+udp://[::1]:8303": {
			"ping_time": 1811999,
			"secret": "42d8f991-f2fa-46e5-a9ae-ebcc93846feb"
		}
	},
	"servers": {
		"42d8f991-f2fa-46e5-a9ae-ebcc93846feb": {
			"info_serial": 0,
			"info": {
				"max_clients": 64,
				"max_players": 64,
				"passworded": false,
				"game_type": "TestDDraceNetwork",
				"name": "My DDNet server",
				"map": {
					"name": "dm1",
					"sha256": "0b0c481d77519c32fbe85624ef16ec0fa9991aec7367ad538bd280f28d8c26cf",
					"size": 5805
				},
				"version": "0.6.4, 16.0.3",
				"clients": []
			}
		}
	}
}
```

`--write-dump` outputs a JSON file compatible with `--read-dump-dir`,
this can be used to synchronize servers across different masterservers.
`--read-dump-dir` is also used to ingest servers from the backward
compatibility layer that pings each server for their server info using
the old protocol.

The `kind` field describe that this is `mastersrv` output and not from a
`backcompat`. This is used for prioritizing `mastersrv` information over
`backcompat` information.

The `now` field contains an integer describing the current time in
milliseconds relative an unspecified epoch that is fixed for each JSON
file. This is done instead of using the current time as the epoch for
better compression of non-changing data.

`secrets` is a map from each server address and to a JSON object
containing the last ping time (`ping_time`) in milliseconds relative to
the same epoch as before, and the server secret (`secret`) that is used
to unify server infos from different addresses of the same logical
server.

`servers` is a map from the aforementioned `secret`s to the
corresponding `info_serial` and `info`.

```json
[
	"tw-0.6+udp://127.0.0.1:8303",
	"tw-0.6+udp://[::1]:8303"
]
```

`--write-addresses` outputs a JSON file containing all addresses
corresponding to servers that are registered to HTTP masterservers. It
does not contain the servers that are obtained via backward
compatibility measures.

This file can be used by an old-style masterserver to also list
new-style servers without the game servers having to register there.

An implementation of this can be found at
https://github.com/heinrich5991/teeworlds/tree/mastersrv_6_backcompat
for Teeworlds 0.5/0.6 masterservers and at
https://github.com/heinrich5991/teeworlds/tree/mastersrv_7_backcompat
for Teeworlds 0.7 masterservers.

All these JSON files can be sent over the network in an efficient way
using https://github.com/heinrich5991/twmaster-collect. It establishes a
zstd-compressed TCP connection authenticated by a string token that is
sent in plain-text. It watches the specified file and transmits it every
time it changes. Due to the zstd-compression, the data sent over the
network is similar to the size of a diff.

Implementation
--------------

The masterserver implementation was done in Rust.

The current gameserver register implementation doesn't support more than
one masterserver for registering.
2022-05-20 08:58:32 +02:00
heinrich5991 4fb050ccb7 Use STUN to determine UDP connectivity and show diagnostics
These diagnostics are supposed to guide the user to problem resolution.
They're displayed if no packet is received from the server within one
second of connecting.

No message if we don't have STUN servers.

"Trying to determine UDP connectivity..." if no answer has been received
from the STUN server yet and it hasn't timed out yet.

"UDP seems to be filtered." if the STUN request has timed out.

"UDP and TCP IP addresses seem to be different. Try disabling VPN,
proxy or network accelerators." if the STUN request has returned an IP
address different from the one obtained via HTTP from info2.ddnet.tw.

"No answer from server yet." otherwise, if the STUN request has returned
no interesting data, indicating that it's likely the game server's
fault.
2022-05-18 09:21:31 +02:00
Chairn 21e64cbdcb Added external linkage instead of static const 2022-04-03 01:12:27 +02:00
Robert Müller 74f87ee604 Remove unnecessary trailing semicolons after method declarations 2022-03-08 20:01:26 +01:00
heinrich5991 a8664a0100 Hide the recvmmsg as an implementation detail 2022-03-04 19:33:06 +01:00
heinrich5991 3211b1de67 Remove unused Flags argument in network code 2022-03-04 19:33:06 +01:00
heinrich5991 471bb441a1 Move NETSOCKET behind a pointer so it can grow 2022-03-04 19:33:06 +01:00
trml f3b0a1e331 Add ClPredictionMargin 2022-01-09 13:08:11 +01:00
Jupeyy a663799188 uint64 -> uint64_t, int64 -> int64_t 2021-06-24 17:19:17 +02:00
Alexander Akulich 2790270abf Adopt upstream refactoring: Mark several functions as 'const' 2021-02-23 18:26:16 +03:00
heinrich5991 7663641ed2 Fix some variable name's style 2020-11-08 17:15:48 +01:00
def 3be8a592e5 Run clang-format
Purely automatic change. In case of conflict with this change, apply the
other change and rerun the formatting to restore it:

$ python scripts/fix_style.py
2020-09-26 21:50:15 +02:00
Andrii cec8bf2195 Read server port from CNetServer but not from app config 2020-08-27 19:54:44 +03:00
Learath 28905c4659 Disable timeout for 0.7 clients 2020-06-20 19:52:23 +03:00
Learath 88ca573682 Serverinfo and Register support for 0.7
Co-authored-by: Tim Schumacher <tim@timakro.de>
2020-06-19 20:28:55 +03:00
Tim Schumacher 442148a194 Begin work on 0.7 support 2020-06-19 20:27:15 +03:00
Learath e9ba23b53a Migrate to 0.6.5 flags 2020-06-19 20:27:15 +03:00
def 85bb376bdc sv_server_info_per_second, sv_van_conn_per_second: allow 0 to disable
Clean up code a bit, also allow larger values
2019-06-11 18:12:43 +02:00
def a116ed3dbe Revert "New try at not decompressing unknown packets"
Causes connection problems with old clients
This reverts commit a3b07dbf9c.
2018-12-30 12:22:05 +01:00
Dennis Felsing 9febf58f37 New try at recvmmsg with some improvements 2018-12-16 22:59:41 +01:00
def a3b07dbf9c New try at not decompressing unknown packets 2018-12-16 22:15:45 +01:00
def d884465710 Revert "Don't decompress packets from unknown IPs"
This reverts commit 9089c2b35a.
2018-12-14 17:12:49 +01:00
Dennis Felsing 9089c2b35a Don't decompress packets from unknown IPs 2018-12-13 16:24:37 +01:00
yangfl 81a39c229b Fix typo 2018-07-10 17:29:02 +08:00
heinrich5991 9454dfbff9 Remove superfluous parameter from CServer::NewClientNoAuthCallback
The `Reset` parameter was only ever set to true, at the only call site.
2018-05-09 23:50:25 +02:00
heinrich5991 1d81d56850 Introduce new, vanilla-compatible server info protocol
This means that we have a reliable and fast way to query for extended info,
while also not wasting network bandwidth.

The protocol is designed to be extensible, there's four bytes space for
encoding more request types (currently zeroed), and there's one string in each
response packet and one string for each player available (currently the empty
string).

The protocol itself has no problems with more than 64 players, although the
current client implementation will drop the player info after the 64th player,
because it uses a static array for storage.

Also fixes #130, the player list is just sorted each time new player info
arrives.
2017-03-29 12:56:13 +02:00
necropotame b8e4e5beea Fix style in several places 2017-03-22 19:45:14 +01:00
east 1cbaa0c4ef copy resendbuffer on timeout protection 2016-05-07 19:28:16 +02:00
def 86a0f46937 Add a connection limit (fixes #408) 2016-04-27 22:09:18 +02:00
east 32ba1c9239 Antispoof dm1 fallback on flood added 2016-04-23 17:23:01 +02:00
oy e91d26b8ac force vital check on essential cl/srv messages and added additional sequence checking 2015-09-01 13:50:45 +02:00
east c1cbfcacf4 correctly reset network connection 2015-08-23 18:12:13 +02:00
east cf9e1373d7 correctly reset client 2015-08-23 17:51:28 +02:00
east c47697bdef session rejoin 2015-08-23 17:01:01 +02:00
east 09166494a5 Antispoof: fix error state 2015-08-23 12:29:41 +02:00
east e9a1c3ef92 vanilla handshake: add vital flags and use correct sequence (support for 0.6.4) 2015-08-15 10:38:39 +02:00
east 76d29f9e23 fixed dublication 2015-08-14 18:40:06 +02:00
east d82b75f760 log/info for security token protocol 2015-08-14 18:39:13 +02:00
east 051ec30832 vanilla handshake: ignore unknown sequence 2015-08-14 18:36:13 +02:00
east 9f1ab9a7bf added simple flooding detection (temprorary) 2015-08-14 18:34:22 +02:00
east d1382349f5 vanilla token handshake implemented 2015-08-14 18:30:49 +02:00
east a2174bf179 anti spoof for token protocol added 2015-08-14 18:30:12 +02:00
east 9d50085782 Fix dummy connect on servers without security token protocol 2015-04-19 23:09:55 +02:00
east 93305aef74 Fix dummy connect delay (Fullfill securitytoken handshake) 2015-04-19 19:53:37 +02:00
def 5dd83b58ff Fix security token conversion in a platform independent way 2015-04-18 23:54:19 +02:00
def a77cd039a6 Get rid of all Clang warnings 2015-04-18 22:29:28 +02:00
eeeee 0dc332c931 fix timeout protection with tokens
timeout protection code now transfers the token properly
2015-03-05 22:33:01 -08:00
heinrich5991 fa0708b99d Make the secure random stuff platform-independent 2015-03-06 01:35:08 +01:00