This allows every game server to provide its own HTTPS server for map
downloads. Since the ingame protocol for downloading map data is very
inefficient, this is desirable. Previously, only servers hosted by DDNet
could benefit from this.
Security concerns:
- Attackers can find out whether a given HTTPS GET request matches a
known answer.
This isn't deemed to be problematic as no cookies for authentication
are sent and only the whole response can be matched.
- Sending requests to honeypot URLs to get people in legal trouble.
This seems to be already possible with HTML image embeds, so it can't
be that bad™.
- Downloading huge files, filling up a player's disk. The players might
cancel when seeing huge files.
There's a generous limit of 1 GiB per map file.
- Downloading huge files transparently compressed with gzip. See above.
Fixes#5812.
/home/deen/isos/ddnet/ddnet-source/src/tools/map_create_pixelart.cpp: In function ‘int InsertPixelArtQuads(CQuad*, int&, const CImageInfo&, const int*, const int*, const bool*)’:
/home/deen/isos/ddnet/ddnet-source/src/tools/map_create_pixelart.cpp:110:9: error: ‘memset’ was not declared in this scope
110 | memset(aVisitedPixels, 0, sizeof(bool) * Img.m_Height * Img.m_Width);
| ^~~~~~
/home/deen/isos/ddnet/ddnet-source/src/tools/map_create_pixelart.cpp:8:1: note: ‘memset’ is defined in header ‘<cstring>’; did you forget to ‘#include <cstring>’?
7 | #include <game/mapitems.h>
+++ |+#include <cstring>
8 |
The `CServer::GetClientVersion` method needs the `version.h` include, so it's moved from the header to the source file, so the include can be removed from the header.
The `GetClientVersion` method is often called with the same `Client != SERVER_DEMO_CLIENT ? GetClientVersion(Client) : CLIENT_VERSIONNR` expression, which also needs the `version.h` include. This expression is moved inside the method, so the include can be removed from all the server entities' and player code.
The `CGameContext::GetClientVersion` method is made a delegate to reduce duplicate code.
The includes of the server entities are also organized further.
The sixup translated player flags were previously applied in `OnDirectInput` directly to the player input data.
However, the input data has already been copied at that point, so predicted inputs were using the incorrect flags.
This is fixed by moving the player flags translation to `OnClientPrepareInput`, which is called first and only once on each new player input before it is copied.
The flags were only being updated based on rather arbitrary conditions, so they could never reset to zero or reduced.
The flags are now always updated. The flags are only used in the output of the `status` command.
4829: Added map_replace_area tool r=def- a=sctt
Motivation:
sometimes mappers need to copy a certain area from a map to another (but also to a different position of the same map).
that's very time consuming on complex maps, as tiles and quads layers have to be manually copied one by one.
map_replace_area is designed to automatically replace all the tiles and quads from a specific area to another.
`Usage: map_replace_area <from_map> <from_x> <from_y> <to_map> <to_x> <to_y> <width> <height> <output_map>`
PoW screenshots attached below.
note: for the moment map_replace_area suppose that you are working on the same map (that's what i needed), that means source and destination maps must have the same layers structure, or else an error is returned.
but i understand it might be useful to also work on totally different maps, in that case users must be able specify layers they want to consider by associating source map layers to destination map layers (by using a config file maybe).
i'm probably going to make another PR to add this enhancement in the future.
PoW:
5343: Add rcon and chat to integration test r=def- a=ChillerDragon
Add more action to the integration test script: chat messages, chat commands, rcon commands.
This unlocks more code coverage at runtime to catch asan issues.
It adds a very restrictive regex on the log format. But I am happy to maintain that in case changes to the log output happen.
The current version is already supporting heinrichs refactor https://github.com/ddnet/ddnet/pull/5036
Also ensure chat messages arrive in the correct format. Would have catched the following issues:
https://github.com/ddnet/ddnet/issues/5342https://github.com/ddnet/ddnet/issues/5340https://github.com/ddnet/ddnet/issues/5302https://github.com/ddnet/ddnet/pull/5126
DEPENDS ON:
https://github.com/ddnet/ddnet/issues/5342https://github.com/ddnet/ddnet/issues/5340
5465: Change from pnglite to libpng for PNG reading r=def- a=heinrich5991
This is desirable mainly because libpng is maintained and pnglite is
not. pnglite was last updated in 2007 (15 years ago) and probably has a
lot of security vulnerabilities.
libpng is an actively maintained library also used by browsers like
Firefox or Chromium, so it's less likely to contain security
vulnerabilities, also it's more likely to be packaged by Linux
distributions.
This also refuses to load images of types not supported by pnglite,
which allows us to think about backward compatibility while also
introducing libpng.
## Checklist
- [x] Tested the change ingame
- [ ] Provided screenshots if it is a visual change
- [ ] Tested in combination with possibly related configuration options
- [ ] Written a unit test if it works standalone, system.c especially
- [ ] Considered possible null pointers and out of bounds array indexing
- [ ] Changed no physics that affect existing maps
- [ ] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssanitizer--undefinedbehavioursanitizer-or-valgrinds-memcheck) (optional)
Co-authored-by: sctt <scottistefano91@gmail.com>
Co-authored-by: term <term@term.sinervis.pri>
Co-authored-by: f <scottistefano91@gmail.com>
Co-authored-by: heinrich5991 <heinrich5991@gmail.com>
Co-authored-by: ChillerDragon <ChillerDragon@gmail.com>
Co-authored-by: def <dennis@felsin9.de>
Co-authored-by: Jupeyy <jupjopjap@gmail.com>
5406: Remove namespace tw r=Jupeyy a=heinrich5991
It didn't have a clear role, it just acted as a distinguisher between
two functions with the same name.
Rename `tw::time_get` to `time_get_nanoseconds` and delete the old
`time_get_nanoseconds`. Move `CCmdlineFix` and the typed
`net_socket_read_wait` function to the global namespace.
## Checklist
- [ ] Tested the change ingame
- [ ] Provided screenshots if it is a visual change
- [ ] Tested in combination with possibly related configuration options
- [ ] Written a unit test if it works standalone, system.c especially
- [ ] Considered possible null pointers and out of bounds array indexing
- [ ] Changed no physics that affect existing maps
- [ ] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssanitizer--undefinedbehavioursanitizer-or-valgrinds-memcheck) (optional)
Co-authored-by: heinrich5991 <heinrich5991@gmail.com>
It didn't have a clear role, it just acted as a distinguisher between
two functions with the same name.
Rename `tw::time_get` to `time_get_nanoseconds` and delete the old
`time_get_nanoseconds`. Move `CCmdlineFix` and the typed
`net_socket_read_wait` function to the global namespace.
Conditional jump or move depends on uninitialised value(s)
at 0x2158DA: int IServer::SendPackMsg<CNetMsg_Sv_Motd, 0>(CNetMsg_Sv_Motd*, int, int) (server.h:73)
by 0x1FE02F: CGameContext::SendMotd(int) (gamecontext.cpp:538)
by 0x20A874: CGameContext::ConchainSpecialMotdupdate(IConsole::IResult*, void*, void (*)(IConsole::IResult*, void*), void*) (gamecontext.cpp:3124)
by 0x145A23: CConsole::Con_Chain(IConsole::IResult*, void*) (console.cpp:1202)
by 0x132086: CConsole::ExecuteLineStroked(int, char const*, int, bool) (console.cpp:528)
by 0x1323A7: CConsole::ExecuteLine(char const*, int, bool) (console.cpp:582)
by 0x1326B1: CConsole::ExecuteFile(char const*, int, bool, int) (console.cpp:625)
by 0x1C60E9: main (server.cpp:3874)
Uninitialised value was created by a heap allocation
at 0x483BE63: operator new(unsigned long) (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
by 0x1C5900: CreateServer() (server.cpp:3755)
by 0x1C5C4C: main (server.cpp:3818)
Also delete it explicitly when `sv_register` is set to register via less
protocols.
On shutdown, wait at most 1 second for the masterserver to respond,
otherwise allow up to 15 seconds.
Fixes#5157.
5264: Prevent users from setting port 1-1023 r=Jupeyy a=def-
Following user report that they set cl_port 1 and it didn't work
<!-- What is the motivation for the changes of this pull request -->
## Checklist
- [x] Tested the change ingame
- [ ] Provided screenshots if it is a visual change
- [ ] Tested in combination with possibly related configuration options
- [ ] Written a unit test if it works standalone, system.c especially
- [ ] Considered possible null pointers and out of bounds array indexing
- [ ] Changed no physics that affect existing maps
- [ ] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssanitizer--undefinedbehavioursanitizer-or-valgrinds-memcheck) (optional)
5266: Use std::make_unique (fixes#5166) r=Jupeyy a=def-
No idea why clang-tidy's modernize-... didn't work
<!-- What is the motivation for the changes of this pull request -->
## Checklist
- [ ] Tested the change ingame
- [ ] Provided screenshots if it is a visual change
- [ ] Tested in combination with possibly related configuration options
- [ ] Written a unit test if it works standalone, system.c especially
- [ ] Considered possible null pointers and out of bounds array indexing
- [ ] Changed no physics that affect existing maps
- [ ] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssanitizer--undefinedbehavioursanitizer-or-valgrinds-memcheck) (optional)
Co-authored-by: def <dennis@felsin9.de>
5252: Use std::vector instead of array, remove base/tl/array.h, algorithm.h, allocator.h, range.h r=def- a=Robyt3
Replaces all usages of `array` with `std::vector`.
I adjusted variable names of variables I changed to use the `v` prefix. Not so in the editor however, as there are already many changes due to clang-tidy enforcing the use of for-each loops.
This allows us to remove all remaining `base/tl` headers except `threading.h`.
Clang-tidy now finds `clang-analyzer-cplusplus.NewDelete` (Use of memory after it is freed), which is also fixed here, though it appears to be a false-positive.
This last remaining usages of `goto` are also removed.
## Checklist
- [X] Tested the change ingame
- [ ] Provided screenshots if it is a visual change
- [ ] Tested in combination with possibly related configuration options
- [ ] Written a unit test if it works standalone, system.c especially
- [ ] Considered possible null pointers and out of bounds array indexing
- [ ] Changed no physics that affect existing maps
- [ ] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssanitizer--undefinedbehavioursanitizer-or-valgrinds-memcheck) (optional)
Co-authored-by: Robert Müller <robytemueller@gmail.com>
5063: Reworked Draggers and Turrets AND show always hooks if they are in the field of view r=def- a=C0D3D3V
Fixes#3622, #4723#4798 and #5054
Here you can find a comparison video with 16.0.3: https://youtu.be/bYwLB1cEMI8
This now also includes the changes from https://github.com/ddnet/ddnet/pull/4980
So please also see my monologue here: https://github.com/ddnet/ddnet/pull/4980
Physic that got changed by this PR:
- Solo players are not included in the calculation of the next team players to the dragger/turrets. This does not affect any map.
- Turrets get switched correctly if players are solo
- Plasma bullets fired by turrets on solo player only explode for solo players and the other way around (except if they unsolo)
- Plasma bullets can no longer be intercepted by other teams
- Turrets can shot now independently at the speed of sv_plasma_per_sec for every team and every solo player
This should be tested by some more players before it get merged!
This reduces the use of snap id's tested here: https://youtu.be/G3nVtdH0--Q
from 16000 to 69
test with the new dragger: https://youtu.be/mzNrDHP7HQs
I did not add the terminal of the server because you basically see no change in the used snap ids
The videos are with cl_predict off
Comparison of the new turrets: https://streamable.com/8us8lk
left old turrets, right the new ones
We should delete ranks on (that where made after 13 Jun 2019):
https://ddnet.tw/maps/Increase-32-Your-32-Speedhttps://ddnet.tw/maps/Fall-32-into-32-the-32-Futurehttps://ddnet.tw/maps/turboSeks
## Checklist
- [x] Tested the change ingame
- [x] Provided screenshots if it is a visual change
- [ ] Tested in combination with possibly related configuration options
- [ ] Written a unit test if it works standalone, system.c especially
- [x] Considered possible null pointers and out of bounds array indexing
- [only on 3 maps] Changed no physics that affect existing maps
- [x] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssanitizer--undefinedbehavioursanitizer-or-valgrinds-memcheck) (optional)
Co-authored-by: c0d3d3v <c0d3d3v@mag-keinen-spam.de>
Makes teehistorian during spawn more reproduceable.
Currently during respawn the first applied input doesn't get recorded.
Always appliying the last sent input fixes this.
Registering
-----------
The idea is that game servers push their server info to the
masterservers every 15 seconds or when the server info changes, but not
more than once per second.
The game servers do not support the old registering protocol anymore,
the backward compatibility is handled by the masterserver.
The register call is a HTTP POST to a URL like
`https://master1.ddnet.tw/ddnet/15/register` and looks like this:
```json
POST /ddnet/15/register HTTP/1.1
Address: tw-0.6+udp://connecting-address.invalid:8303
Secret: 81fa3955-6f83-4290-818d-31c0906b1118
Challenge-Secret: 81fa3955-6f83-4290-818d-31c0906b1118:tw0.6/ipv6
Info-Serial: 0
{
"max_clients": 64,
"max_players": 64,
"passworded": false,
"game_type": "TestDDraceNetwork",
"name": "My DDNet server",
"map": {
"name": "dm1",
"sha256": "0b0c481d77519c32fbe85624ef16ec0fa9991aec7367ad538bd280f28d8c26cf",
"size": 5805
},
"version": "0.6.4, 16.0.3",
"clients": []
}
```
The `Address` header declares that the server wants to register itself as
a `tw-0.6+udp` server, i.e. a server speaking a Teeworlds-0.6-compatible
protocol.
The free-form `Secret` header is used as a server identity, the server
list will be deduplicated via this secret.
The free-form `Challenge-Secret` is sent back via UDP for a port forward
check. This might have security implications as the masterserver can be
asked to send a UDP packet containing some user-controlled bytes. This
is somewhat mitigated by the fact that it can only go to an
attacker-controlled IP address.
The `Info-Serial` header is an integer field that should increase each
time the server info (in the body) changes. The masterserver uses that
field to ensure that it doesn't use old server infos.
The body is a free-form JSON object set by the game server. It should
contain certain keys in the correct form to be accepted by clients. The
body is optional if the masterserver already confirmed the reception of
the info with the given `Info-Serial`.
Not shown in this payload is the `Connless-Token` header that is used
for Teeworlds 0.7 style communication.
Also not shown is the `Challenge-Token` that should be included once the
server receives the challenge token via UDP.
The masterserver responds with a `200 OK` with a body like this:
```
{"status":"success"}
```
The `status` field can be `success` if the server was successfully
registered on the masterserver, `need_challenge` if the masterserver
wants the correct `Challenge-Token` header before the register process
is successful, `need_info` if the server sent an empty body but the
masterserver doesn't actually know the server info.
It can also be `error` if the request was malformed, only in this case
an HTTP status code except `200 OK` is sent.
Synchronization
---------------
The masterserver keeps state and outputs JSON files every second.
```json
{
"servers": [
{
"addresses": [
"tw-0.6+udp://127.0.0.1:8303",
"tw-0.6+udp://[::1]:8303"
],
"info_serial": 0,
"info": {
"max_clients": 64,
"max_players": 64,
"passworded": false,
"game_type": "TestDDraceNetwork",
"name": "My DDNet server",
"map": {
"name": "dm1",
"sha256": "0b0c481d77519c32fbe85624ef16ec0fa9991aec7367ad538bd280f28d8c26cf",
"size": 5805
},
"version": "0.6.4, 16.0.3",
"clients": []
}
}
]
}
```
`servers.json` (or configured by `--out`) is a server list that is
compatible with DDNet 15.5+ clients. It is a JSON object containing a
single key `servers` with a list of game servers. Each game server is
represented by a JSON object with an `addresses` key containing a list
of all known addresses of the server and an `info` key containing the
free-form server info sent by the game server. The free-form `info` JSON
object re-encoded by the master server and thus canonicalized and
stripped of any whitespace characters outside strings.
```json
{
"kind": "mastersrv",
"now": 1816002,
"secrets": {
"tw-0.6+udp://127.0.0.1:8303": {
"ping_time": 1811999,
"secret": "42d8f991-f2fa-46e5-a9ae-ebcc93846feb"
},
"tw-0.6+udp://[::1]:8303": {
"ping_time": 1811999,
"secret": "42d8f991-f2fa-46e5-a9ae-ebcc93846feb"
}
},
"servers": {
"42d8f991-f2fa-46e5-a9ae-ebcc93846feb": {
"info_serial": 0,
"info": {
"max_clients": 64,
"max_players": 64,
"passworded": false,
"game_type": "TestDDraceNetwork",
"name": "My DDNet server",
"map": {
"name": "dm1",
"sha256": "0b0c481d77519c32fbe85624ef16ec0fa9991aec7367ad538bd280f28d8c26cf",
"size": 5805
},
"version": "0.6.4, 16.0.3",
"clients": []
}
}
}
}
```
`--write-dump` outputs a JSON file compatible with `--read-dump-dir`,
this can be used to synchronize servers across different masterservers.
`--read-dump-dir` is also used to ingest servers from the backward
compatibility layer that pings each server for their server info using
the old protocol.
The `kind` field describe that this is `mastersrv` output and not from a
`backcompat`. This is used for prioritizing `mastersrv` information over
`backcompat` information.
The `now` field contains an integer describing the current time in
milliseconds relative an unspecified epoch that is fixed for each JSON
file. This is done instead of using the current time as the epoch for
better compression of non-changing data.
`secrets` is a map from each server address and to a JSON object
containing the last ping time (`ping_time`) in milliseconds relative to
the same epoch as before, and the server secret (`secret`) that is used
to unify server infos from different addresses of the same logical
server.
`servers` is a map from the aforementioned `secret`s to the
corresponding `info_serial` and `info`.
```json
[
"tw-0.6+udp://127.0.0.1:8303",
"tw-0.6+udp://[::1]:8303"
]
```
`--write-addresses` outputs a JSON file containing all addresses
corresponding to servers that are registered to HTTP masterservers. It
does not contain the servers that are obtained via backward
compatibility measures.
This file can be used by an old-style masterserver to also list
new-style servers without the game servers having to register there.
An implementation of this can be found at
https://github.com/heinrich5991/teeworlds/tree/mastersrv_6_backcompat
for Teeworlds 0.5/0.6 masterservers and at
https://github.com/heinrich5991/teeworlds/tree/mastersrv_7_backcompat
for Teeworlds 0.7 masterservers.
All these JSON files can be sent over the network in an efficient way
using https://github.com/heinrich5991/twmaster-collect. It establishes a
zstd-compressed TCP connection authenticated by a string token that is
sent in plain-text. It watches the specified file and transmits it every
time it changes. Due to the zstd-compression, the data sent over the
network is similar to the size of a diff.
Implementation
--------------
The masterserver implementation was done in Rust.
The current gameserver register implementation doesn't support more than
one masterserver for registering.
ChillerDragon
