Add `IJob::Abortable(bool)` function which jobs can call to specify whether they can be aborted. Jobs are not abortable per default. Abortable jobs may have their state set to `IJob::STATE_ABORTED` at any point if the job was aborted. The job state should be checked periodically in the `IJob::Run` function and the job should terminate at the earliest, safe opportunity when aborted. Scheduled jobs which are not abortable are guaranteed to fully complete before the job pool is shut down. However, if the job pool is already shutting down, no additional jobs will be enqueue anymore and abortable jobs will immediately be aborted.
In particular, the sound loading, community icon loading, master chooser and host lookup jobs are specified as being abortable. Conversely, the jobs saving replay demos, editor maps and screenshots are expected to finish before the client is shut down.
When the client is quitting/restarting, it will now disconnect from the current server first, before saving the config, to ensure that any actions that happen on disconnect (demo recorders being stopped etc.) happen first. The shutdown message is rendered before disconnecting and waiting for background jobs to finish.
The HTTP client is now initialized later during server launch, after the network initialization. Error handling is added and the server stops if the HTTP client could not be initialized, same as the client.
The `RunBlocking` functions are removed, as they are not used anymore after curl-multi was added.
The function `IJob::Status` is renamed to `State` and `IJob::STATE_PENDING` is renamed to `STATE_QUEUED` for consistency with naming of the HTTP client.
The member variables of the engine interface are encapsulated and the `jobs.h` include is removed from `engine.h`, which removes transitive includes of `system.h`.
Documentation for all job and job pool API is added.
Instead of sending the message from the first client which is not ingame, choose the first client which is fully empty, to make it less likely that the message is sent from a currently connecting client.
Move all code for handling of config variables from console to config manager. The console no longer depends on the config manager, instead the config manager now depends on the console.
Add `struct`s to manage config variables of different types (int, color and string). The config manager now keeps a list of all config variables, so usage of the preprocessor can be avoided except for code to initially create all config variables. Additionally, a separate list of just the game config variables (config variables with `CFGFLAG_GAME`) is kept to optimize the `ResetGameSettings` function, as this function is called whenever connecting to a server and should be fast. Previously, this function was even less efficient because it preformed a linear search for every individual game config variable to find the respective command data.
Move console commands that opperate only on config variables (`reset`, `toggle` and `+toggle`) to config manager. Ensure that these commands only opperate on the desired config variables (client or server, respectively) by checking `IConsole::FlagMask`.
Add `IConfigManager::SetReadOnly` function to set/unset config variables as read-only instead of manually overriding the command handlers for the `sv_rescue` and `sv_test_cmds` config variables. This also fixes that read-only config variables could still be changed by using the `reset` command. A console message is now printed when trying to change a read-only config variable. Removing the special handling for these two config variables is additionally useful so the console does not need to keep a pointer to config values and manager.
Use a `CHeap` for the config variables, their help texts and the previous values of string config variables to avoid many separate allocations as well usage of static variables. Also use the heap to store the unknown commands instead of using `std::string`s.
Properly trigger command chain when resetting config variables with the `reset` command and when resetting game settings on map loading. Closes#7461.
Format default value for color variables as RGB/RGBA hex with dollar sign prefix. Closes#5523.
Add log message when using `reset` with a variable that does not exist. Use `log_error` instead of `dbg_msg` when saving config file fails.
Support unlimited number of config save callbacks instead of at most 16. The code also becomes more readable by using an `std::vector` instead of a fixed-size array and a separate num variable.
Consistently name `MACRO_CONFIG_*` parameters when declaring the macros.
Add `IConsole::CMDLINE_LENGTH` constant to represent the maximum length of the console input and thereby reduce usage of magic numbers for buffer sizes.
Apply the name bans system also to player clans, meaning players joining with banned clan names are kicked and changing the clan to a banned name while ingame has no effect.
Additionally, trim UTF-8 whitespace from beginning and end of clan. This was already done for player names but not for clans.
Closes#7516.
Unify all code for name bans in new class `CNameBans` in the existing `name_ban.cpp/h` files. The previously global function `IsNameBanned` is now the member function `CNameBans::IsBanned`. The existing name ban tests are extended for the `CNameBans` class.
Move `CNameBan` constructor definition to source file to avoid including `system.h` in the header file. Use `bool` instead of `int` for `m_IsSubstring`. Reorder `CNameBan` constructor arguments and remove unnecessary default value.
Call expected server callback functions to simulate clients dynamically connecting and disconnecting when changing the `dbg_dummies` variable. This makes the debug dummies more useful for debugging. Previously, the debug dummies were considered invalid clients, whereas they are now considered to be ingame, so they should behave mostly like real clients being connected to the server. The debug dummies also have correct client names now, e.g. "Debug dummy 42".
The game server code is cleaned up by moving all special handling for debug dummies to the engine server function `CServer::UpdateDebugDummies`.
The left/right direction inputs for debug dummies are now properly added to the client input array, so their input handling should be consistent with normal clients, which fixes some inconsistent prediction with debug dummies.
`stdout_output_level` for printing to stdout, `console_output_level` for
printing to local console and remote console and `loglevel` for the log
file.
Keep the old log level filters 0 for info and more severe, 1 for debug
and more severe and 2 for trace and more severe, introducing -1 for
warn, and -2 for error. -3 will show no log messages at all.
Use `std::vector` in cases where elements are only inserted at the end of the collection.
Use `std::deque` in cases where elements are only inserted/deleted at the beginning/end of the collection.
Use `std::list` in the remaining single case where elements are being removed from arbitrary positions and added at either the beginning or the end of the collection.
Adjust variables names. Don't use separate prefix for `std::deque`s and `std::list`s, as they are only used very rarely. Closes#6779.
Reimplement the Linux FIFO file server and client controls on Windows by using Named Pipes.
The DDNet server/client acts as a named pipe server and receives messages.
Messages can be posted to the named pipe server by connecting to it as a client.
The named pipe client can for instance be controlled from the command line with PowerShell.
The PowerShell script `scripts/send_named_pipe.ps1` is added for this purpose.
For example the PowerShell command `./send_named_pipe.ps1 "testpipe" "echo a"` sends the command `echo a` to the pipe named `testpipe`.
Multiple commands can be sent at the same time by separating them with semicolons or newlines.
The `CServer::GetClientVersion` method needs the `version.h` include, so it's moved from the header to the source file, so the include can be removed from the header.
The `GetClientVersion` method is often called with the same `Client != SERVER_DEMO_CLIENT ? GetClientVersion(Client) : CLIENT_VERSIONNR` expression, which also needs the `version.h` include. This expression is moved inside the method, so the include can be removed from all the server entities' and player code.
The `CGameContext::GetClientVersion` method is made a delegate to reduce duplicate code.
The includes of the server entities are also organized further.
5252: Use std::vector instead of array, remove base/tl/array.h, algorithm.h, allocator.h, range.h r=def- a=Robyt3
Replaces all usages of `array` with `std::vector`.
I adjusted variable names of variables I changed to use the `v` prefix. Not so in the editor however, as there are already many changes due to clang-tidy enforcing the use of for-each loops.
This allows us to remove all remaining `base/tl` headers except `threading.h`.
Clang-tidy now finds `clang-analyzer-cplusplus.NewDelete` (Use of memory after it is freed), which is also fixed here, though it appears to be a false-positive.
This last remaining usages of `goto` are also removed.
## Checklist
- [X] Tested the change ingame
- [ ] Provided screenshots if it is a visual change
- [ ] Tested in combination with possibly related configuration options
- [ ] Written a unit test if it works standalone, system.c especially
- [ ] Considered possible null pointers and out of bounds array indexing
- [ ] Changed no physics that affect existing maps
- [ ] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssanitizer--undefinedbehavioursanitizer-or-valgrinds-memcheck) (optional)
Co-authored-by: Robert Müller <robytemueller@gmail.com>
Registering
-----------
The idea is that game servers push their server info to the
masterservers every 15 seconds or when the server info changes, but not
more than once per second.
The game servers do not support the old registering protocol anymore,
the backward compatibility is handled by the masterserver.
The register call is a HTTP POST to a URL like
`https://master1.ddnet.tw/ddnet/15/register` and looks like this:
```json
POST /ddnet/15/register HTTP/1.1
Address: tw-0.6+udp://connecting-address.invalid:8303
Secret: 81fa3955-6f83-4290-818d-31c0906b1118
Challenge-Secret: 81fa3955-6f83-4290-818d-31c0906b1118:tw0.6/ipv6
Info-Serial: 0
{
"max_clients": 64,
"max_players": 64,
"passworded": false,
"game_type": "TestDDraceNetwork",
"name": "My DDNet server",
"map": {
"name": "dm1",
"sha256": "0b0c481d77519c32fbe85624ef16ec0fa9991aec7367ad538bd280f28d8c26cf",
"size": 5805
},
"version": "0.6.4, 16.0.3",
"clients": []
}
```
The `Address` header declares that the server wants to register itself as
a `tw-0.6+udp` server, i.e. a server speaking a Teeworlds-0.6-compatible
protocol.
The free-form `Secret` header is used as a server identity, the server
list will be deduplicated via this secret.
The free-form `Challenge-Secret` is sent back via UDP for a port forward
check. This might have security implications as the masterserver can be
asked to send a UDP packet containing some user-controlled bytes. This
is somewhat mitigated by the fact that it can only go to an
attacker-controlled IP address.
The `Info-Serial` header is an integer field that should increase each
time the server info (in the body) changes. The masterserver uses that
field to ensure that it doesn't use old server infos.
The body is a free-form JSON object set by the game server. It should
contain certain keys in the correct form to be accepted by clients. The
body is optional if the masterserver already confirmed the reception of
the info with the given `Info-Serial`.
Not shown in this payload is the `Connless-Token` header that is used
for Teeworlds 0.7 style communication.
Also not shown is the `Challenge-Token` that should be included once the
server receives the challenge token via UDP.
The masterserver responds with a `200 OK` with a body like this:
```
{"status":"success"}
```
The `status` field can be `success` if the server was successfully
registered on the masterserver, `need_challenge` if the masterserver
wants the correct `Challenge-Token` header before the register process
is successful, `need_info` if the server sent an empty body but the
masterserver doesn't actually know the server info.
It can also be `error` if the request was malformed, only in this case
an HTTP status code except `200 OK` is sent.
Synchronization
---------------
The masterserver keeps state and outputs JSON files every second.
```json
{
"servers": [
{
"addresses": [
"tw-0.6+udp://127.0.0.1:8303",
"tw-0.6+udp://[::1]:8303"
],
"info_serial": 0,
"info": {
"max_clients": 64,
"max_players": 64,
"passworded": false,
"game_type": "TestDDraceNetwork",
"name": "My DDNet server",
"map": {
"name": "dm1",
"sha256": "0b0c481d77519c32fbe85624ef16ec0fa9991aec7367ad538bd280f28d8c26cf",
"size": 5805
},
"version": "0.6.4, 16.0.3",
"clients": []
}
}
]
}
```
`servers.json` (or configured by `--out`) is a server list that is
compatible with DDNet 15.5+ clients. It is a JSON object containing a
single key `servers` with a list of game servers. Each game server is
represented by a JSON object with an `addresses` key containing a list
of all known addresses of the server and an `info` key containing the
free-form server info sent by the game server. The free-form `info` JSON
object re-encoded by the master server and thus canonicalized and
stripped of any whitespace characters outside strings.
```json
{
"kind": "mastersrv",
"now": 1816002,
"secrets": {
"tw-0.6+udp://127.0.0.1:8303": {
"ping_time": 1811999,
"secret": "42d8f991-f2fa-46e5-a9ae-ebcc93846feb"
},
"tw-0.6+udp://[::1]:8303": {
"ping_time": 1811999,
"secret": "42d8f991-f2fa-46e5-a9ae-ebcc93846feb"
}
},
"servers": {
"42d8f991-f2fa-46e5-a9ae-ebcc93846feb": {
"info_serial": 0,
"info": {
"max_clients": 64,
"max_players": 64,
"passworded": false,
"game_type": "TestDDraceNetwork",
"name": "My DDNet server",
"map": {
"name": "dm1",
"sha256": "0b0c481d77519c32fbe85624ef16ec0fa9991aec7367ad538bd280f28d8c26cf",
"size": 5805
},
"version": "0.6.4, 16.0.3",
"clients": []
}
}
}
}
```
`--write-dump` outputs a JSON file compatible with `--read-dump-dir`,
this can be used to synchronize servers across different masterservers.
`--read-dump-dir` is also used to ingest servers from the backward
compatibility layer that pings each server for their server info using
the old protocol.
The `kind` field describe that this is `mastersrv` output and not from a
`backcompat`. This is used for prioritizing `mastersrv` information over
`backcompat` information.
The `now` field contains an integer describing the current time in
milliseconds relative an unspecified epoch that is fixed for each JSON
file. This is done instead of using the current time as the epoch for
better compression of non-changing data.
`secrets` is a map from each server address and to a JSON object
containing the last ping time (`ping_time`) in milliseconds relative to
the same epoch as before, and the server secret (`secret`) that is used
to unify server infos from different addresses of the same logical
server.
`servers` is a map from the aforementioned `secret`s to the
corresponding `info_serial` and `info`.
```json
[
"tw-0.6+udp://127.0.0.1:8303",
"tw-0.6+udp://[::1]:8303"
]
```
`--write-addresses` outputs a JSON file containing all addresses
corresponding to servers that are registered to HTTP masterservers. It
does not contain the servers that are obtained via backward
compatibility measures.
This file can be used by an old-style masterserver to also list
new-style servers without the game servers having to register there.
An implementation of this can be found at
https://github.com/heinrich5991/teeworlds/tree/mastersrv_6_backcompat
for Teeworlds 0.5/0.6 masterservers and at
https://github.com/heinrich5991/teeworlds/tree/mastersrv_7_backcompat
for Teeworlds 0.7 masterservers.
All these JSON files can be sent over the network in an efficient way
using https://github.com/heinrich5991/twmaster-collect. It establishes a
zstd-compressed TCP connection authenticated by a string token that is
sent in plain-text. It watches the specified file and transmits it every
time it changes. Due to the zstd-compression, the data sent over the
network is similar to the size of a diff.
Implementation
--------------
The masterserver implementation was done in Rust.
The current gameserver register implementation doesn't support more than
one masterserver for registering.
This makes the "black console window" less important on Windows (or
anywhere else, for that matter), lets you see logs from other threads in
the f1 console, and removes the distinction between `IConsole::Print`
and `dbg_msg`.
4726: Display super with "super" string in scoreboard (fixes#4724) r=heinrich5991 a=def-
instead of showing the number 64
Alternative to https://github.com/ddnet/ddnet/pull/4725
<!-- What is the motivation for the changes of this pull request -->
## Checklist
- [ ] Tested the change ingame
- [ ] Provided screenshots if it is a visual change
- [ ] Tested in combination with possibly related configuration options
- [ ] Written a unit test if it works standalone, system.c especially
- [ ] Considered possible null pointers and out of bounds array indexing
- [ ] Changed no physics that affect existing maps
- [ ] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssanitizer--undefinedbehavioursanitizer-or-valgrinds-memcheck) (optional)
4727: Increase MAX_IDS to 32k (fixes#4723) r=heinrich5991 a=def-
Not sure I like it, otherwise we have to edit Game City.
Maybe we should fail this in map editor already instead of allowing to
create a map which requires > MAX_IDS
<!-- What is the motivation for the changes of this pull request -->
## Checklist
- [x] Tested the change ingame
- [ ] Provided screenshots if it is a visual change
- [ ] Tested in combination with possibly related configuration options
- [ ] Written a unit test if it works standalone, system.c especially
- [ ] Considered possible null pointers and out of bounds array indexing
- [ ] Changed no physics that affect existing maps
- [ ] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssanitizer--undefinedbehavioursanitizer-or-valgrinds-memcheck) (optional)
4728: Implement shields that remove specific weapon (fixes#4572) r=heinrich5991 a=def-
Thanks to Cellegen for graphics.
Functionality works, still a bit buggy sprite display (see screenshot), maybe someone can
quickly spot what I'm doing wrong
<img width="898" alt="Screenshot 2022-02-18 at 00 50 14" src="https://user-images.githubusercontent.com/2335377/154591108-d89cb475-daaf-4a6a-9748-17160a1e71a2.png">
## Checklist
- [x] Tested the change ingame
- [x] Provided screenshots if it is a visual change
- [ ] Tested in combination with possibly related configuration options
- [ ] Written a unit test if it works standalone, system.c especially
- [ ] Considered possible null pointers and out of bounds array indexing
- [ ] Changed no physics that affect existing maps
- [ ] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssanitizer--undefinedbehavioursanitizer-or-valgrinds-memcheck) (optional)
4880: Add mysql to github workflow in ubuntu 20.04 r=heinrich5991 a=def-
<!-- What is the motivation for the changes of this pull request -->
## Checklist
- [ ] Tested the change ingame
- [ ] Provided screenshots if it is a visual change
- [ ] Tested in combination with possibly related configuration options
- [ ] Written a unit test if it works standalone, system.c especially
- [ ] Considered possible null pointers and out of bounds array indexing
- [ ] Changed no physics that affect existing maps
- [ ] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssanitizer--undefinedbehavioursanitizer-or-valgrinds-memcheck) (optional)
5022: Use std::clamp r=heinrich5991 a=def-
Not sure if we want to, there are some easy functions like this we could
take from stdlib now
<!-- What is the motivation for the changes of this pull request -->
## Checklist
- [ ] Tested the change ingame
- [ ] Provided screenshots if it is a visual change
- [ ] Tested in combination with possibly related configuration options
- [ ] Written a unit test if it works standalone, system.c especially
- [ ] Considered possible null pointers and out of bounds array indexing
- [ ] Changed no physics that affect existing maps
- [ ] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssanitizer--undefinedbehavioursanitizer-or-valgrinds-memcheck) (optional)
Co-authored-by: Dennis Felsing <dennis@felsin9.de>
Co-authored-by: Jupeyy <jupjopjap@gmail.com>
- Replace hardcoded constants (namely '2') with enum value.
- Use the map type constants instead of bool-to-int casts.
This is a step toward support of more map types: e.g. we can have smaller
map versions with some mapres omitted for newer DDNet client versions.
Not sure I like it, otherwise we have to edit Game City.
Maybe we should fail this in map editor already instead of allowing to
create a map which requires > MAX_IDS
4456: Pull some more refactoring from the upstream r=def- a=Kaffeine
The main motivation is making it easier to take (and push) patches to/from `teeworlds`
## Checklist
- [x] Tested the change ingame
- [ ] Provided screenshots if it is a visual change
- [ ] Tested in combination with possibly related configuration options
- [ ] Written a unit test if it works standalone, system.c especially
- [ ] Considered possible null pointers and out of bounds array indexing
- [ ] Changed no physics that affect existing maps
- [ ] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssanitizer--undefinedbehavioursanitizer-or-valgrinds-memcheck) (optional)
I'm not that familiar with the code. Please let me know if this commit make sense or not:
56900d7644
IIUC two other commits from https://github.com/teeworlds/teeworlds/pull/2594 are not relevant for DDNet (because we're reducing the CPU load differently).
Co-authored-by: Alexander Akulich <akulichalexander@gmail.com>
Co-authored-by: Robert Müller <robert.mueller@student.uni-siegen.de>
ChillerDragon
Edgar
