Commit graph

65 commits

Author SHA1 Message Date
ChillerDragon fd437e0f76 Apache RequestHeader needs the headers module 2023-04-01 16:35:09 +02:00
heinrich5991 7bacbb0ba7 Add README.md for the mastersrv, briefly explaining the setup 2022-12-21 13:11:57 +01:00
heinrich5991 dcd76fd3e1 Add support for Rust code in DDNet
The glue is done using the [cxx crate](https://cxx.rs/) on the Rust
side.

As a proof-of-concept, only a small console command (`rust_version`)
printing the currently used Rust version was added.

You can generate and open the Rust documentation using
`DDNET_TEST_NO_LINK=1 cargo doc --open`.

You can run the Rust tests using `cmake --build <build dir> --target
run_rust_tests`, they're automatically included in the `run_tests`
target as well.

Rust tests don't work on Windows in debug mode on Windows because Rust
cannot currently link with the debug version of the C stdlib on Windows:
https://github.com/rust-lang/rust/issues/39016.

---

The stuff in `src/rust-bridge` is generated using
```
cxxbridge src/engine/shared/rust_version.rs --output src/rust-bridge/engine/shared/rust_version.cpp --output src/rust-bridge/engine/shared/rust_version.h
cxxbridge src/engine/console.rs --output src/rust-bridge/cpp/console.cpp --output src/rust-bridge/cpp/console.h
```
2022-10-19 23:46:06 +02:00
Edgar 5cfec69fad fix import on mastersrv 2022-10-07 21:32:53 +02:00
heinrich5991 135577b7f6 mastersrv: Small convenience function 2022-06-13 17:00:54 +02:00
heinrich5991 ce08cc0e53 mastersrv: Add Action: delete to /ddnet/15/register
This will delete a server from the server list without a timeout,
allowing servers that want to stop registering or servers that shut down
gracefully to no longer appear in the server list.
2022-06-03 11:27:47 +02:00
heinrich5991 ee8896d014 Make mastersrv compile with Rust 1.48.0 2022-05-20 16:29:01 +02:00
heinrich5991 6b65ccb945 Add HTTP masterserver registering and HTTP masterserver
Registering
-----------

The idea is that game servers push their server info to the
masterservers every 15 seconds or when the server info changes, but not
more than once per second.

The game servers do not support the old registering protocol anymore,
the backward compatibility is handled by the masterserver.

The register call is a HTTP POST to a URL like
`https://master1.ddnet.tw/ddnet/15/register` and looks like this:
```json
POST /ddnet/15/register HTTP/1.1
Address: tw-0.6+udp://connecting-address.invalid:8303
Secret: 81fa3955-6f83-4290-818d-31c0906b1118
Challenge-Secret: 81fa3955-6f83-4290-818d-31c0906b1118:tw0.6/ipv6
Info-Serial: 0

{
	"max_clients": 64,
	"max_players": 64,
	"passworded": false,
	"game_type": "TestDDraceNetwork",
	"name": "My DDNet server",
	"map": {
		"name": "dm1",
		"sha256": "0b0c481d77519c32fbe85624ef16ec0fa9991aec7367ad538bd280f28d8c26cf",
		"size": 5805
	},
	"version": "0.6.4, 16.0.3",
	"clients": []
}
```

The `Address` header declares that the server wants to register itself as
a `tw-0.6+udp` server, i.e. a server speaking a Teeworlds-0.6-compatible
protocol.

The free-form `Secret` header is used as a server identity, the server
list will be deduplicated via this secret.

The free-form `Challenge-Secret` is sent back via UDP for a port forward
check.  This might have security implications as the masterserver can be
asked to send a UDP packet containing some user-controlled bytes. This
is somewhat mitigated by the fact that it can only go to an
attacker-controlled IP address.

The `Info-Serial` header is an integer field that should increase each
time the server info (in the body) changes. The masterserver uses that
field to ensure that it doesn't use old server infos.

The body is a free-form JSON object set by the game server. It should
contain certain keys in the correct form to be accepted by clients. The
body is optional if the masterserver already confirmed the reception of
the info with the given `Info-Serial`.

Not shown in this payload is the `Connless-Token` header that is used
for Teeworlds 0.7 style communication.

Also not shown is the `Challenge-Token` that should be included once the
server receives the challenge token via UDP.

The masterserver responds with a `200 OK` with a body like this:

```
{"status":"success"}
```

The `status` field can be `success` if the server was successfully
registered on the masterserver, `need_challenge` if the masterserver
wants the correct `Challenge-Token` header before the register process
is successful, `need_info` if the server sent an empty body but the
masterserver doesn't actually know the server info.

It can also be `error` if the request was malformed, only in this case
an HTTP status code except `200 OK` is sent.

Synchronization
---------------

The masterserver keeps state and outputs JSON files every second.

```json
{
	"servers": [
		{
			"addresses": [
				"tw-0.6+udp://127.0.0.1:8303",
				"tw-0.6+udp://[::1]:8303"
			],
			"info_serial": 0,
			"info": {
				"max_clients": 64,
				"max_players": 64,
				"passworded": false,
				"game_type": "TestDDraceNetwork",
				"name": "My DDNet server",
				"map": {
					"name": "dm1",
					"sha256": "0b0c481d77519c32fbe85624ef16ec0fa9991aec7367ad538bd280f28d8c26cf",
					"size": 5805
				},
				"version": "0.6.4, 16.0.3",
				"clients": []
			}
		}
	]
}
```

`servers.json` (or configured by `--out`) is a server list that is
compatible with DDNet 15.5+ clients. It is a JSON object containing a
single key `servers` with a list of game servers. Each game server is
represented by a JSON object with an `addresses` key containing a list
of all known addresses of the server and an `info` key containing the
free-form server info sent by the game server. The free-form `info` JSON
object re-encoded by the master server and thus canonicalized and
stripped of any whitespace characters outside strings.

```json
{
	"kind": "mastersrv",
	"now": 1816002,
	"secrets": {
		"tw-0.6+udp://127.0.0.1:8303": {
			"ping_time": 1811999,
			"secret": "42d8f991-f2fa-46e5-a9ae-ebcc93846feb"
		},
		"tw-0.6+udp://[::1]:8303": {
			"ping_time": 1811999,
			"secret": "42d8f991-f2fa-46e5-a9ae-ebcc93846feb"
		}
	},
	"servers": {
		"42d8f991-f2fa-46e5-a9ae-ebcc93846feb": {
			"info_serial": 0,
			"info": {
				"max_clients": 64,
				"max_players": 64,
				"passworded": false,
				"game_type": "TestDDraceNetwork",
				"name": "My DDNet server",
				"map": {
					"name": "dm1",
					"sha256": "0b0c481d77519c32fbe85624ef16ec0fa9991aec7367ad538bd280f28d8c26cf",
					"size": 5805
				},
				"version": "0.6.4, 16.0.3",
				"clients": []
			}
		}
	}
}
```

`--write-dump` outputs a JSON file compatible with `--read-dump-dir`,
this can be used to synchronize servers across different masterservers.
`--read-dump-dir` is also used to ingest servers from the backward
compatibility layer that pings each server for their server info using
the old protocol.

The `kind` field describe that this is `mastersrv` output and not from a
`backcompat`. This is used for prioritizing `mastersrv` information over
`backcompat` information.

The `now` field contains an integer describing the current time in
milliseconds relative an unspecified epoch that is fixed for each JSON
file. This is done instead of using the current time as the epoch for
better compression of non-changing data.

`secrets` is a map from each server address and to a JSON object
containing the last ping time (`ping_time`) in milliseconds relative to
the same epoch as before, and the server secret (`secret`) that is used
to unify server infos from different addresses of the same logical
server.

`servers` is a map from the aforementioned `secret`s to the
corresponding `info_serial` and `info`.

```json
[
	"tw-0.6+udp://127.0.0.1:8303",
	"tw-0.6+udp://[::1]:8303"
]
```

`--write-addresses` outputs a JSON file containing all addresses
corresponding to servers that are registered to HTTP masterservers. It
does not contain the servers that are obtained via backward
compatibility measures.

This file can be used by an old-style masterserver to also list
new-style servers without the game servers having to register there.

An implementation of this can be found at
https://github.com/heinrich5991/teeworlds/tree/mastersrv_6_backcompat
for Teeworlds 0.5/0.6 masterservers and at
https://github.com/heinrich5991/teeworlds/tree/mastersrv_7_backcompat
for Teeworlds 0.7 masterservers.

All these JSON files can be sent over the network in an efficient way
using https://github.com/heinrich5991/twmaster-collect. It establishes a
zstd-compressed TCP connection authenticated by a string token that is
sent in plain-text. It watches the specified file and transmits it every
time it changes. Due to the zstd-compression, the data sent over the
network is similar to the size of a diff.

Implementation
--------------

The masterserver implementation was done in Rust.

The current gameserver register implementation doesn't support more than
one masterserver for registering.
2022-05-20 08:58:32 +02:00
Jupeyy 3c597aff95 Switch to nanoseconds 2022-05-18 18:05:41 +02:00
heinrich5991 fa4bcd5ec0 Unify logging infrastructure between IConsole and dbg_msg
This makes the "black console window" less important on Windows (or
anywhere else, for that matter), lets you see logs from other threads in
the f1 console, and removes the distinction between `IConsole::Print`
and `dbg_msg`.
2022-04-29 15:21:26 +02:00
def 790c1cc0aa Switch new players to DDNet config dir, support Teeworlds as fallback
This way new players will get DDNet directory, old ones can switch
directory if they want, or keep using the old one.

If we ever enforce a switch in a future version, this will make it
easier since older DDNet versions will also support the DDNet directory
already.
2022-04-25 09:50:45 +02:00
Chairn ae1e3c079d clang-format 2022-04-03 01:12:27 +02:00
Chairn 21e64cbdcb Added external linkage instead of static const 2022-04-03 01:12:27 +02:00
heinrich5991 3211b1de67 Remove unused Flags argument in network code 2022-03-04 19:33:06 +01:00
Robert Müller 89f126cf8d Remove obsolete ignore_convention comment 2022-02-16 20:54:11 +01:00
Dennis Felsing e615d4b844 Fix tools 2022-02-15 01:07:26 +01:00
Robert Müller f20443b694 fix command line arguments with unicode on windows
Co-authored-by: heinrich5991 <heinrich5991@gmail.com>
2021-12-06 19:48:25 +01:00
Jupeyy a663799188 uint64 -> uint64_t, int64 -> int64_t 2021-06-24 17:19:17 +02:00
Alexander Akulich c2f276cee1 Port CConfig API from the upstream (0.7.5)
The old (g_Config) API is kept to not break the stuff.

See commits:
    de5859b371
    78076761eb
2021-01-10 17:10:19 +03:00
Alexander Akulich 09c05ab99f Port CConsole::Init() from the upstream code
Partial port of upstream commit c0bf37fb48
2021-01-10 16:51:18 +03:00
def 3be8a592e5 Run clang-format
Purely automatic change. In case of conflict with this change, apply the
other change and rerun the formatting to restore it:

$ python scripts/fix_style.py
2020-09-26 21:50:15 +02:00
def 6846a6ecad Fixes and stress improvements 2018-12-17 20:05:50 +01:00
heinrich5991 1d81d56850 Introduce new, vanilla-compatible server info protocol
This means that we have a reliable and fast way to query for extended info,
while also not wasting network bandwidth.

The protocol is designed to be extensible, there's four bytes space for
encoding more request types (currently zeroed), and there's one string in each
response packet and one string for each player available (currently the empty
string).

The protocol itself has no problems with more than 64 players, although the
current client implementation will drop the player info after the 64th player,
because it uses a static array for storage.

Also fixes #130, the player list is just sorted each time new player info
arrives.
2017-03-29 12:56:13 +02:00
def 61e47422f6 Clean up debug messages a bit (lower case, no useless errors) 2016-05-02 21:35:32 +02:00
def 6e8ada1ac8 Fix whitespaces a bit 2015-07-09 02:12:35 +02:00
def a77cd039a6 Get rid of all Clang warnings 2015-04-18 22:29:28 +02:00
def bd62b5da79 Clean up a few compile warnings 2014-12-31 13:04:31 +01:00
def 3607218f12 Merge ddrace64 from eeeeee 2013-12-31 06:13:57 +01:00
oy 1a62770a56 fixed banning on the master server 2013-02-24 17:35:31 +01:00
oy 13d06e45ac fixed used nettype in server browser and try to use ipv4 and ipv6 socket when using a bindaddr. Closes #940 2013-02-24 17:26:59 +01:00
oy 4073520214 cleaned up content folder creation. Closes #845 2012-01-09 01:38:45 +01:00
oy 29738552a6 made bindaddr config work for client and master too. Closes #909 2012-01-06 19:17:14 +01:00
oy b44ee3d975 reworked ban system 2011-12-29 23:36:53 +01:00
oy fede33447a fixed problem with the master server 2011-04-15 20:38:06 +02:00
oy 06115dd49d added "Whitespace and line Endings cleanup" by GreYFoX 2011-04-13 20:37:12 +02:00
oy b6118ae9fe mapped IPv4 addresses within master servers server list to IPv6 to decrease the size of an entry and increased the servers per packet 2011-04-08 23:56:15 +02:00
m!nus 63c46b3e56 changed how server types are distinguished and fixed a few bugs 2011-04-02 17:39:12 +02:00
m!nus 3ebdc2a4ce added downward compatibility to mastersrv for 0.5 (and earlier) 2011-04-02 17:35:50 +02:00
oy 59d56cd332 finished IPv6 support (master servers aren't compatible with 0.5 and older). Closes #233 2011-03-30 12:08:33 +02:00
oy 0ae19e1339 removed support for old server infos 2011-03-04 18:14:08 +01:00
m!nus 3eb1c64428 Fixed ban adding in mastersrv (check for existing ban) 2011-01-23 17:54:32 +01:00
m!nus 05d274444a Add banning to mastersrv: any packet from a banned IP (and optionally a fixed port) will be dropped (also affects clients. Put your bans into master.cfg: "ban 127.0.0.1" or "ban 127.0.0.1:8303". It's reloaded every 5 minutes. This change also adds IStorage, IConsole and IKernel to mastersrv. 2011-01-23 17:54:27 +01:00
Sworddragon fc9211c777 Updated copyrights 2010-11-20 21:26:06 +01:00
Magnus Auvinen 72c06a2589 copied refactor to trunk 2010-05-29 07:25:38 +00:00
Magnus Auvinen 8acf5ed788 upgraded the master server code so it can handle more servers 2008-11-15 13:29:56 +00:00
Magnus Auvinen 7ebdb8d4d7 fixed logging 2008-10-01 18:34:43 +00:00
Magnus Auvinen 8d99a3d3c9 fixed so old request methods work aswell 2008-09-04 17:25:55 +00:00
Magnus Auvinen 53da3f0d40 added favorites. no saving of them yet however 2008-09-03 20:03:01 +00:00
Magnus Auvinen 62d9ff05d6 fixed connection less packets. they behave like the old version so version server and master servers still work 2008-08-31 19:29:09 +00:00
Magnus Auvinen 817f431377 moved out e_system, e_detect, vmath and math to a little base library 2008-08-14 17:19:13 +00:00