5252: Use std::vector instead of array, remove base/tl/array.h, algorithm.h, allocator.h, range.h r=def- a=Robyt3
Replaces all usages of `array` with `std::vector`.
I adjusted variable names of variables I changed to use the `v` prefix. Not so in the editor however, as there are already many changes due to clang-tidy enforcing the use of for-each loops.
This allows us to remove all remaining `base/tl` headers except `threading.h`.
Clang-tidy now finds `clang-analyzer-cplusplus.NewDelete` (Use of memory after it is freed), which is also fixed here, though it appears to be a false-positive.
This last remaining usages of `goto` are also removed.
## Checklist
- [X] Tested the change ingame
- [ ] Provided screenshots if it is a visual change
- [ ] Tested in combination with possibly related configuration options
- [ ] Written a unit test if it works standalone, system.c especially
- [ ] Considered possible null pointers and out of bounds array indexing
- [ ] Changed no physics that affect existing maps
- [ ] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssanitizer--undefinedbehavioursanitizer-or-valgrinds-memcheck) (optional)
Co-authored-by: Robert Müller <robytemueller@gmail.com>
Registering
-----------
The idea is that game servers push their server info to the
masterservers every 15 seconds or when the server info changes, but not
more than once per second.
The game servers do not support the old registering protocol anymore,
the backward compatibility is handled by the masterserver.
The register call is a HTTP POST to a URL like
`https://master1.ddnet.tw/ddnet/15/register` and looks like this:
```json
POST /ddnet/15/register HTTP/1.1
Address: tw-0.6+udp://connecting-address.invalid:8303
Secret: 81fa3955-6f83-4290-818d-31c0906b1118
Challenge-Secret: 81fa3955-6f83-4290-818d-31c0906b1118:tw0.6/ipv6
Info-Serial: 0
{
"max_clients": 64,
"max_players": 64,
"passworded": false,
"game_type": "TestDDraceNetwork",
"name": "My DDNet server",
"map": {
"name": "dm1",
"sha256": "0b0c481d77519c32fbe85624ef16ec0fa9991aec7367ad538bd280f28d8c26cf",
"size": 5805
},
"version": "0.6.4, 16.0.3",
"clients": []
}
```
The `Address` header declares that the server wants to register itself as
a `tw-0.6+udp` server, i.e. a server speaking a Teeworlds-0.6-compatible
protocol.
The free-form `Secret` header is used as a server identity, the server
list will be deduplicated via this secret.
The free-form `Challenge-Secret` is sent back via UDP for a port forward
check. This might have security implications as the masterserver can be
asked to send a UDP packet containing some user-controlled bytes. This
is somewhat mitigated by the fact that it can only go to an
attacker-controlled IP address.
The `Info-Serial` header is an integer field that should increase each
time the server info (in the body) changes. The masterserver uses that
field to ensure that it doesn't use old server infos.
The body is a free-form JSON object set by the game server. It should
contain certain keys in the correct form to be accepted by clients. The
body is optional if the masterserver already confirmed the reception of
the info with the given `Info-Serial`.
Not shown in this payload is the `Connless-Token` header that is used
for Teeworlds 0.7 style communication.
Also not shown is the `Challenge-Token` that should be included once the
server receives the challenge token via UDP.
The masterserver responds with a `200 OK` with a body like this:
```
{"status":"success"}
```
The `status` field can be `success` if the server was successfully
registered on the masterserver, `need_challenge` if the masterserver
wants the correct `Challenge-Token` header before the register process
is successful, `need_info` if the server sent an empty body but the
masterserver doesn't actually know the server info.
It can also be `error` if the request was malformed, only in this case
an HTTP status code except `200 OK` is sent.
Synchronization
---------------
The masterserver keeps state and outputs JSON files every second.
```json
{
"servers": [
{
"addresses": [
"tw-0.6+udp://127.0.0.1:8303",
"tw-0.6+udp://[::1]:8303"
],
"info_serial": 0,
"info": {
"max_clients": 64,
"max_players": 64,
"passworded": false,
"game_type": "TestDDraceNetwork",
"name": "My DDNet server",
"map": {
"name": "dm1",
"sha256": "0b0c481d77519c32fbe85624ef16ec0fa9991aec7367ad538bd280f28d8c26cf",
"size": 5805
},
"version": "0.6.4, 16.0.3",
"clients": []
}
}
]
}
```
`servers.json` (or configured by `--out`) is a server list that is
compatible with DDNet 15.5+ clients. It is a JSON object containing a
single key `servers` with a list of game servers. Each game server is
represented by a JSON object with an `addresses` key containing a list
of all known addresses of the server and an `info` key containing the
free-form server info sent by the game server. The free-form `info` JSON
object re-encoded by the master server and thus canonicalized and
stripped of any whitespace characters outside strings.
```json
{
"kind": "mastersrv",
"now": 1816002,
"secrets": {
"tw-0.6+udp://127.0.0.1:8303": {
"ping_time": 1811999,
"secret": "42d8f991-f2fa-46e5-a9ae-ebcc93846feb"
},
"tw-0.6+udp://[::1]:8303": {
"ping_time": 1811999,
"secret": "42d8f991-f2fa-46e5-a9ae-ebcc93846feb"
}
},
"servers": {
"42d8f991-f2fa-46e5-a9ae-ebcc93846feb": {
"info_serial": 0,
"info": {
"max_clients": 64,
"max_players": 64,
"passworded": false,
"game_type": "TestDDraceNetwork",
"name": "My DDNet server",
"map": {
"name": "dm1",
"sha256": "0b0c481d77519c32fbe85624ef16ec0fa9991aec7367ad538bd280f28d8c26cf",
"size": 5805
},
"version": "0.6.4, 16.0.3",
"clients": []
}
}
}
}
```
`--write-dump` outputs a JSON file compatible with `--read-dump-dir`,
this can be used to synchronize servers across different masterservers.
`--read-dump-dir` is also used to ingest servers from the backward
compatibility layer that pings each server for their server info using
the old protocol.
The `kind` field describe that this is `mastersrv` output and not from a
`backcompat`. This is used for prioritizing `mastersrv` information over
`backcompat` information.
The `now` field contains an integer describing the current time in
milliseconds relative an unspecified epoch that is fixed for each JSON
file. This is done instead of using the current time as the epoch for
better compression of non-changing data.
`secrets` is a map from each server address and to a JSON object
containing the last ping time (`ping_time`) in milliseconds relative to
the same epoch as before, and the server secret (`secret`) that is used
to unify server infos from different addresses of the same logical
server.
`servers` is a map from the aforementioned `secret`s to the
corresponding `info_serial` and `info`.
```json
[
"tw-0.6+udp://127.0.0.1:8303",
"tw-0.6+udp://[::1]:8303"
]
```
`--write-addresses` outputs a JSON file containing all addresses
corresponding to servers that are registered to HTTP masterservers. It
does not contain the servers that are obtained via backward
compatibility measures.
This file can be used by an old-style masterserver to also list
new-style servers without the game servers having to register there.
An implementation of this can be found at
https://github.com/heinrich5991/teeworlds/tree/mastersrv_6_backcompat
for Teeworlds 0.5/0.6 masterservers and at
https://github.com/heinrich5991/teeworlds/tree/mastersrv_7_backcompat
for Teeworlds 0.7 masterservers.
All these JSON files can be sent over the network in an efficient way
using https://github.com/heinrich5991/twmaster-collect. It establishes a
zstd-compressed TCP connection authenticated by a string token that is
sent in plain-text. It watches the specified file and transmits it every
time it changes. Due to the zstd-compression, the data sent over the
network is similar to the size of a diff.
Implementation
--------------
The masterserver implementation was done in Rust.
The current gameserver register implementation doesn't support more than
one masterserver for registering.
This makes the "black console window" less important on Windows (or
anywhere else, for that matter), lets you see logs from other threads in
the f1 console, and removes the distinction between `IConsole::Print`
and `dbg_msg`.
This way new players will get DDNet directory, old ones can switch
directory if they want, or keep using the old one.
If we ever enforce a switch in a future version, this will make it
easier since older DDNet versions will also support the DDNet directory
already.
> Warn when a local variable shadows another local variable or parameter.
Found one actual bug in graphics_threaded.cpp
Should reduce confusion in the future when reading source code
Previously, only Linux used an internal buffer (for optimized
receiving). Now all OSs use an internal buffer so that the call to
`net_udp_recv` behaves the same on all platforms.
3197: 0.7 map converter - Stop embedding rgb images r=heinrich5991 a=Patiga
DDNet doesn't CMapItemTypeImage version 2, meaning it doesn't have
support yet for having rgb images in maps. Currently, trying that would result in
invalid map files.
## Checklist
- [ ] Tested the change ingame
- [ ] Provided screenshots if it is a visual change
- [ ] Tested in combination with possibly related configuration options
- [ ] Written a unit test if it works standalone, system.c especially
- [x] Considered possible null pointers and out of bounds array indexing
- [x] Changed no physics that affect existing maps
- [ ] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssanitizer--undefinedbehavioursanitizer-or-valgrinds-memcheck) (optional)
Co-authored-by: Patiga <dev@patiga.eu>
DDNet doesn't CMapItemTypeImage version 2, meaning it doesn't have
support yet for having rgb images in maps. Currently, trying that would result in
invalid map files.
/media/ddnet/src/game/client/animstate.cpp:54:27: warning: The left operand of '*' is a garbage value [clang-analyzer-core.UndefinedBinaryOperatorResult]
pSeq->m_X += pAdded->m_X * Amount;
^
and mark some false positives
/media/ddnet/src/engine/client/backend_sdl.cpp:329:30: warning: Call to 'malloc' has an allocation size of 0 bytes [clang-analyzer-optin.portability.UnixAPI]
and print a more telling error message
I'll check all our existing maps for failures now:
$ find . -iname '*.map' | while read i; do echo $i; map_extract $i > /dev/null; find . -iname '*.png' | while read j; do dilate $j; rm -- $j; done; done
For skins Soreu and the skin DB team is already taking care.
Purely automatic change. In case of conflict with this change, apply the
other change and rerun the formatting to restore it:
$ python scripts/fix_style.py
2789: Refresh server browser immediately when updated ddnet info arrived r=heinrich5991 a=def-
At the moment it only happens the next time you press refresh.
Before we had it so that it always reloaded once the ddnet info arrived,
causing a quick flickering and lots of packets having to be resent every
time someone pressed refresh, even if nothing changed.
The new approach combines the advantages of both without the
disadvantages. An even nicer way would be to compare the json objects,
so that news and version updates don't matter, but our json library
doesn't seem to support that.
2797: dilate: Don't crash if file doesn't exist r=heinrich5991 a=def-
Co-authored-by: def <dennis@felsin9.de>
2301: map_convert_07: Check image width for divisibility by 16 r=heinrich5991 a=def-
As noticed on Sunny Land.
> [5:31 PM] redix: in theory it should be enough if width and height are divisible by 16
Now running this over our maps and fixing them manually.
Co-authored-by: def <dennis@felsin9.de>
SHA256 was chosen because it is reasonably standard, the file names
don't explode in length (this rules out SHA512) and it is supported by
basically all versions of OpenSSL (this rules out SHA512/256 and SHA3).
The protocol is changed in a backward compatible way: The supporting
server sends the SHA256 corresponding to the map in the `MAP_DETAILS`
message prior to sending the `MAP_CHANGE` message. The client saves the
SHA256 obtained from the `MAP_DETAILS` message until the next
`MAP_CHANGE` message.
For servers not supporting this protocol, the client falls back to
simply opening maps like in the previous scheme.
Remove the `map_version` tool, it is not being used and would have been
a little bit effort to update.
Use the OpenSSL implementation of SHA256 if it is supported, otherwise
fall back to a public domain one.
Fix#1127.
Replace `mem_free` by `free`, and `mem_alloc` by `malloc` or `calloc`
(the latter one being used to allocate a zeroed array of elements,
sometimes, this makes a call to `mem_zero` superfluous).
This results in having to remove `mem_stats` which previously recorded
the number of allocations and their size that the Teeworlds code did
directly.
Remove OOM handling in `src/game/client/components/binds.cpp`.
Remove needless copying in the CSV code in
`src/game/client/components/statboard.cpp`.
Also annotate `dbg_msg`, `str_format` and `str_timestamp_ex` so that the
compiler can determine whether the format strings are correct.
Fix the compiler warnings generated by these extra warnings -- some of
them were security issues.
This system can easily be extended by independent authors without
collisions, something the old system with plain increasing integers did
not allow.
Do this by utilizing the previously unused message code `NETMSG_NULL`
which has a value of 0.
This works for engine and game messages, snapshot items and events.
See UTS#39 "Unicode Security Mechanisms":
http://www.unicode.org/reports/tr39/
This means that characters with accents or other things around them are
now considered the confusable with the base character.
Fixes#557. Fixes#575.
ChillerDragon
