ddnet/src/engine/client/serverbrowser.h
heinrich5991 6b65ccb945 Add HTTP masterserver registering and HTTP masterserver
Registering
-----------

The idea is that game servers push their server info to the
masterservers every 15 seconds or when the server info changes, but not
more than once per second.

The game servers do not support the old registering protocol anymore,
the backward compatibility is handled by the masterserver.

The register call is a HTTP POST to a URL like
`https://master1.ddnet.tw/ddnet/15/register` and looks like this:
```json
POST /ddnet/15/register HTTP/1.1
Address: tw-0.6+udp://connecting-address.invalid:8303
Secret: 81fa3955-6f83-4290-818d-31c0906b1118
Challenge-Secret: 81fa3955-6f83-4290-818d-31c0906b1118:tw0.6/ipv6
Info-Serial: 0

{
	"max_clients": 64,
	"max_players": 64,
	"passworded": false,
	"game_type": "TestDDraceNetwork",
	"name": "My DDNet server",
	"map": {
		"name": "dm1",
		"sha256": "0b0c481d77519c32fbe85624ef16ec0fa9991aec7367ad538bd280f28d8c26cf",
		"size": 5805
	},
	"version": "0.6.4, 16.0.3",
	"clients": []
}
```

The `Address` header declares that the server wants to register itself as
a `tw-0.6+udp` server, i.e. a server speaking a Teeworlds-0.6-compatible
protocol.

The free-form `Secret` header is used as a server identity, the server
list will be deduplicated via this secret.

The free-form `Challenge-Secret` is sent back via UDP for a port forward
check.  This might have security implications as the masterserver can be
asked to send a UDP packet containing some user-controlled bytes. This
is somewhat mitigated by the fact that it can only go to an
attacker-controlled IP address.

The `Info-Serial` header is an integer field that should increase each
time the server info (in the body) changes. The masterserver uses that
field to ensure that it doesn't use old server infos.

The body is a free-form JSON object set by the game server. It should
contain certain keys in the correct form to be accepted by clients. The
body is optional if the masterserver already confirmed the reception of
the info with the given `Info-Serial`.

Not shown in this payload is the `Connless-Token` header that is used
for Teeworlds 0.7 style communication.

Also not shown is the `Challenge-Token` that should be included once the
server receives the challenge token via UDP.

The masterserver responds with a `200 OK` with a body like this:

```
{"status":"success"}
```

The `status` field can be `success` if the server was successfully
registered on the masterserver, `need_challenge` if the masterserver
wants the correct `Challenge-Token` header before the register process
is successful, `need_info` if the server sent an empty body but the
masterserver doesn't actually know the server info.

It can also be `error` if the request was malformed, only in this case
an HTTP status code except `200 OK` is sent.

Synchronization
---------------

The masterserver keeps state and outputs JSON files every second.

```json
{
	"servers": [
		{
			"addresses": [
				"tw-0.6+udp://127.0.0.1:8303",
				"tw-0.6+udp://[::1]:8303"
			],
			"info_serial": 0,
			"info": {
				"max_clients": 64,
				"max_players": 64,
				"passworded": false,
				"game_type": "TestDDraceNetwork",
				"name": "My DDNet server",
				"map": {
					"name": "dm1",
					"sha256": "0b0c481d77519c32fbe85624ef16ec0fa9991aec7367ad538bd280f28d8c26cf",
					"size": 5805
				},
				"version": "0.6.4, 16.0.3",
				"clients": []
			}
		}
	]
}
```

`servers.json` (or configured by `--out`) is a server list that is
compatible with DDNet 15.5+ clients. It is a JSON object containing a
single key `servers` with a list of game servers. Each game server is
represented by a JSON object with an `addresses` key containing a list
of all known addresses of the server and an `info` key containing the
free-form server info sent by the game server. The free-form `info` JSON
object re-encoded by the master server and thus canonicalized and
stripped of any whitespace characters outside strings.

```json
{
	"kind": "mastersrv",
	"now": 1816002,
	"secrets": {
		"tw-0.6+udp://127.0.0.1:8303": {
			"ping_time": 1811999,
			"secret": "42d8f991-f2fa-46e5-a9ae-ebcc93846feb"
		},
		"tw-0.6+udp://[::1]:8303": {
			"ping_time": 1811999,
			"secret": "42d8f991-f2fa-46e5-a9ae-ebcc93846feb"
		}
	},
	"servers": {
		"42d8f991-f2fa-46e5-a9ae-ebcc93846feb": {
			"info_serial": 0,
			"info": {
				"max_clients": 64,
				"max_players": 64,
				"passworded": false,
				"game_type": "TestDDraceNetwork",
				"name": "My DDNet server",
				"map": {
					"name": "dm1",
					"sha256": "0b0c481d77519c32fbe85624ef16ec0fa9991aec7367ad538bd280f28d8c26cf",
					"size": 5805
				},
				"version": "0.6.4, 16.0.3",
				"clients": []
			}
		}
	}
}
```

`--write-dump` outputs a JSON file compatible with `--read-dump-dir`,
this can be used to synchronize servers across different masterservers.
`--read-dump-dir` is also used to ingest servers from the backward
compatibility layer that pings each server for their server info using
the old protocol.

The `kind` field describe that this is `mastersrv` output and not from a
`backcompat`. This is used for prioritizing `mastersrv` information over
`backcompat` information.

The `now` field contains an integer describing the current time in
milliseconds relative an unspecified epoch that is fixed for each JSON
file. This is done instead of using the current time as the epoch for
better compression of non-changing data.

`secrets` is a map from each server address and to a JSON object
containing the last ping time (`ping_time`) in milliseconds relative to
the same epoch as before, and the server secret (`secret`) that is used
to unify server infos from different addresses of the same logical
server.

`servers` is a map from the aforementioned `secret`s to the
corresponding `info_serial` and `info`.

```json
[
	"tw-0.6+udp://127.0.0.1:8303",
	"tw-0.6+udp://[::1]:8303"
]
```

`--write-addresses` outputs a JSON file containing all addresses
corresponding to servers that are registered to HTTP masterservers. It
does not contain the servers that are obtained via backward
compatibility measures.

This file can be used by an old-style masterserver to also list
new-style servers without the game servers having to register there.

An implementation of this can be found at
https://github.com/heinrich5991/teeworlds/tree/mastersrv_6_backcompat
for Teeworlds 0.5/0.6 masterservers and at
https://github.com/heinrich5991/teeworlds/tree/mastersrv_7_backcompat
for Teeworlds 0.7 masterservers.

All these JSON files can be sent over the network in an efficient way
using https://github.com/heinrich5991/twmaster-collect. It establishes a
zstd-compressed TCP connection authenticated by a string token that is
sent in plain-text. It watches the specified file and transmits it every
time it changes. Due to the zstd-compression, the data sent over the
network is similar to the size of a diff.

Implementation
--------------

The masterserver implementation was done in Rust.

The current gameserver register implementation doesn't support more than
one masterserver for registering.
2022-05-20 08:58:32 +02:00

238 lines
6.6 KiB
C++

/* (c) Magnus Auvinen. See licence.txt in the root of the distribution for more information. */
/* If you are missing that file, acquire a complete release at teeworlds.com. */
#ifndef ENGINE_CLIENT_SERVERBROWSER_H
#define ENGINE_CLIENT_SERVERBROWSER_H
#include <engine/config.h>
#include <engine/console.h>
#include <engine/external/json-parser/json.h>
#include <engine/serverbrowser.h>
#include <engine/shared/config.h>
#include <engine/shared/http.h>
#include <engine/shared/memheap.h>
class IServerBrowserHttp;
class IServerBrowserPingCache;
class CServerBrowser : public IServerBrowser
{
public:
class CServerEntry
{
public:
NETADDR m_Addr;
int64_t m_RequestTime;
bool m_RequestIgnoreInfo;
int m_GotInfo;
bool m_Request64Legacy;
CServerInfo m_Info;
CServerEntry *m_pNextIp; // ip hashed list
CServerEntry *m_pPrevReq; // request list
CServerEntry *m_pNextReq;
};
struct CNetworkCountry
{
enum
{
MAX_SERVERS = 1024
};
char m_aName[256];
int m_FlagID;
NETADDR m_aServers[MAX_SERVERS];
char m_aTypes[MAX_SERVERS][32];
int m_NumServers;
void Reset()
{
m_NumServers = 0;
m_FlagID = -1;
m_aName[0] = '\0';
};
/*void Add(NETADDR Addr, char* pType) {
if (m_NumServers < MAX_SERVERS)
{
m_aServers[m_NumServers] = Addr;
str_copy(m_aTypes[m_NumServers], pType, sizeof(m_aTypes[0]));
m_NumServers++;
}
};*/
};
enum
{
MAX_FAVORITES = 2048,
MAX_COUNTRIES = 32,
MAX_TYPES = 32,
};
struct CNetwork
{
CNetworkCountry m_aCountries[MAX_COUNTRIES];
int m_NumCountries;
char m_aTypes[MAX_TYPES][32];
int m_NumTypes;
};
CServerBrowser();
virtual ~CServerBrowser();
// interface functions
void Refresh(int Type) override;
bool IsRefreshing() const override;
bool IsGettingServerlist() const override;
int LoadingProgression() const override;
int NumServers() const override { return m_NumServers; }
int Players(const CServerInfo &Item) const override
{
return g_Config.m_BrFilterSpectators ? Item.m_NumPlayers : Item.m_NumClients;
}
int Max(const CServerInfo &Item) const override
{
return g_Config.m_BrFilterSpectators ? Item.m_MaxPlayers : Item.m_MaxClients;
}
int NumSortedServers() const override { return m_NumSortedServers; }
const CServerInfo *SortedGet(int Index) const override;
bool GotInfo(const NETADDR &Addr) const override;
bool IsFavorite(const NETADDR &Addr) const override;
bool IsFavoritePingAllowed(const NETADDR &Addr) const override;
void AddFavorite(const NETADDR &Addr) override;
void FavoriteAllowPing(const NETADDR &Addr, bool AllowPing) override;
void RemoveFavorite(const NETADDR &Addr) override;
const char *GetTutorialServer() override;
void LoadDDNetRanks();
void RecheckOfficial();
void LoadDDNetServers();
void LoadDDNetInfoJson();
const json_value *LoadDDNetInfo();
int HasRank(const char *pMap);
int NumCountries(int Network) override { return m_aNetworks[Network].m_NumCountries; }
int GetCountryFlag(int Network, int Index) override { return m_aNetworks[Network].m_aCountries[Index].m_FlagID; }
const char *GetCountryName(int Network, int Index) override { return m_aNetworks[Network].m_aCountries[Index].m_aName; }
int NumTypes(int Network) override { return m_aNetworks[Network].m_NumTypes; }
const char *GetType(int Network, int Index) override { return m_aNetworks[Network].m_aTypes[Index]; }
void DDNetFilterAdd(char *pFilter, const char *pName) override;
void DDNetFilterRem(char *pFilter, const char *pName) override;
bool DDNetFiltered(char *pFilter, const char *pName) override;
void CountryFilterClean(int Network) override;
void TypeFilterClean(int Network) override;
//
void Update(bool ForceResort);
void Set(const NETADDR &Addr, int Type, int Token, const CServerInfo *pInfo);
void RequestCurrentServer(const NETADDR &Addr) const;
void RequestCurrentServerWithRandomToken(const NETADDR &Addr, int *pBasicToken, int *pToken) const;
void SetCurrentServerPing(const NETADDR &Addr, int Ping);
void SetBaseInfo(class CNetClient *pClient, const char *pNetVersion);
void OnInit();
void RequestImpl64(const NETADDR &Addr, CServerEntry *pEntry) const;
void QueueRequest(CServerEntry *pEntry);
CServerEntry *Find(const NETADDR &Addr);
int GetCurrentType() override { return m_ServerlistType; }
private:
CNetClient *m_pNetClient;
class IConsole *m_pConsole;
class IEngine *m_pEngine;
class IFriends *m_pFriends;
class IStorage *m_pStorage;
char m_aNetVersion[128];
bool m_RefreshingHttp = false;
IServerBrowserHttp *m_pHttp = nullptr;
IServerBrowserPingCache *m_pPingCache = nullptr;
const char *m_pHttpPrevBestUrl = nullptr;
CHeap m_ServerlistHeap;
CServerEntry **m_ppServerlist;
int *m_pSortedServerlist;
NETADDR m_aFavoriteServers[MAX_FAVORITES];
bool m_aFavoriteServersAllowPing[MAX_FAVORITES];
int m_NumFavoriteServers;
CNetwork m_aNetworks[NUM_NETWORKS];
int m_OwnLocation = CServerInfo::LOC_UNKNOWN;
json_value *m_pDDNetInfo;
CServerEntry *m_aServerlistIp[256]; // ip hash list
CServerEntry *m_pFirstReqServer; // request list
CServerEntry *m_pLastReqServer;
int m_NumRequests;
// used instead of g_Config.br_max_requests to get more servers
int m_CurrentMaxRequests;
int m_NeedRefresh;
int m_NumSortedServers;
int m_NumSortedServersCapacity;
int m_NumServers;
int m_NumServerCapacity;
int m_Sorthash;
char m_aFilterString[64];
char m_aFilterGametypeString[128];
int m_ServerlistType;
int64_t m_BroadcastTime;
unsigned char m_aTokenSeed[16];
bool m_SortOnNextUpdate;
int FindFavorite(const NETADDR &Addr) const;
int GenerateToken(const NETADDR &Addr) const;
static int GetBasicToken(int Token);
static int GetExtraToken(int Token);
// sorting criteria
bool SortCompareName(int Index1, int Index2) const;
bool SortCompareMap(int Index1, int Index2) const;
bool SortComparePing(int Index1, int Index2) const;
bool SortCompareGametype(int Index1, int Index2) const;
bool SortCompareNumPlayers(int Index1, int Index2) const;
bool SortCompareNumClients(int Index1, int Index2) const;
bool SortCompareNumPlayersAndPing(int Index1, int Index2) const;
//
void Filter();
void Sort();
int SortHash() const;
void CleanUp();
void UpdateFromHttp();
CServerEntry *Add(const NETADDR &Addr);
void RemoveRequest(CServerEntry *pEntry);
void RequestImpl(const NETADDR &Addr, CServerEntry *pEntry, int *pBasicToken, int *pToken, bool RandomToken) const;
void RegisterCommands();
static void Con_LeakIpAddress(IConsole::IResult *pResult, void *pUserData);
void SetInfo(CServerEntry *pEntry, const CServerInfo &Info);
void SetLatency(NETADDR Addr, int Latency);
static void ConfigSaveCallback(IConfigManager *pConfigManager, void *pUserData);
};
#endif