Commit graph

29 commits

Author SHA1 Message Date
heinrich5991 c5cd8d354d Add a new failure log level to the HTTP module
Use it for the "determine best masterserver" job, so that we can debug
failures.
2021-06-15 00:12:06 +02:00
bors[bot] 7e9b0fcb6b
Merge #3858
3858: Reduce spamminess of http server browser r=heinrich5991 a=def-

As reported by fokkonaut on Discord. Not sure if we lose relevant
information.

Previously:
```
[2021-05-28 12:08:54][http]: http https://master2.ddnet.tw/ddnet/15/servers.json
[2021-05-28 12:08:55][http]: task done https://master2.ddnet.tw/ddnet/15/servers.json
[2021-05-28 12:08:55][http]: http https://master2.ddnet.tw/ddnet/15/servers.json
[2021-05-28 12:08:57][http]: task done https://master2.ddnet.tw/ddnet/15/servers.json
[2021-05-28 12:08:57][serverbrowse_http]: found master, url='https://master2.ddnet.tw/ddnet/15/servers.json' time=2437ms
[2021-05-28 12:08:57][http]: http https://master4.ddnet.tw/ddnet/15/servers.json
[2021-05-28 12:08:57][http]: task failed. libcurl error: Could not resolve host: master4.ddnet.tw
[2021-05-28 12:08:57][http]: http https://master3.ddnet.tw/ddnet/15/servers.json
[2021-05-28 12:08:57][http]: task failed. libcurl error: Could not resolve host: master3.ddnet.tw
[2021-05-28 12:08:57][http]: http https://master1.ddnet.tw/ddnet/15/servers.json
[2021-05-28 12:08:57][http]: task done https://master1.ddnet.tw/ddnet/15/servers.json
[2021-05-28 12:08:57][http]: http https://master1.ddnet.tw/ddnet/15/servers.json
[2021-05-28 12:08:58][http]: task done https://master1.ddnet.tw/ddnet/15/servers.json
[2021-05-28 12:08:58][serverbrowse_http]: found master, url='https://master1.ddnet.tw/ddnet/15/servers.json' time=201ms
[2021-05-28 12:08:58][serverbrowse_http]: determined best master, url='https://master1.ddnet.tw/ddnet/15/servers.json' time=201ms
[2021-05-28 12:08:58][http]: http https://master1.ddnet.tw/ddnet/15/servers.json
[2021-05-28 12:08:58][http]: task done https://master1.ddnet.tw/ddnet/15/servers.json
```
Now:
```
[2021-05-28 12:29:58][http]: http https://master1.ddnet.tw/ddnet/15/servers.json
[2021-05-28 12:29:59][http]: task done https://master1.ddnet.tw/ddnet/15/servers.json
[2021-05-28 12:30:00][serverbrowse_http]: found master, url='https://master2.ddnet.tw/ddnet/15/servers.json' time=799ms
[2021-05-28 12:30:00][serverbrowse_http]: found master, url='https://master1.ddnet.tw/ddnet/15/servers.json' time=43ms
[2021-05-28 12:30:00][serverbrowse_http]: determined best master, url='https://master1.ddnet.tw/ddnet/15/servers.json' time=43ms
```
<!-- What is the motivation for the changes of this pull request -->

## Checklist

- [x] Tested the change ingame
- [ ] Provided screenshots if it is a visual change
- [ ] Tested in combination with possibly related configuration options
- [ ] Written a unit test if it works standalone, system.c especially
- [ ] Considered possible null pointers and out of bounds array indexing
- [ ] Changed no physics that affect existing maps
- [ ] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssanitizer--undefinedbehavioursanitizer-or-valgrinds-memcheck) (optional)


Co-authored-by: def <dennis@felsin9.de>
2021-05-28 15:13:42 +00:00
def bcdb3e0247 Reduce spamminess of http server browser
As reported by fokkonaut on Discord. Not sure if we lose relevant
information.

Previously:

[2021-05-28 12:08:54][http]: http https://master2.ddnet.tw/ddnet/15/servers.json
[2021-05-28 12:08:55][http]: task done https://master2.ddnet.tw/ddnet/15/servers.json
[2021-05-28 12:08:55][http]: http https://master2.ddnet.tw/ddnet/15/servers.json
[2021-05-28 12:08:57][http]: task done https://master2.ddnet.tw/ddnet/15/servers.json
[2021-05-28 12:08:57][serverbrowse_http]: found master, url='https://master2.ddnet.tw/ddnet/15/servers.json' time=2437ms
[2021-05-28 12:08:57][http]: http https://master4.ddnet.tw/ddnet/15/servers.json
[2021-05-28 12:08:57][http]: task failed. libcurl error: Could not resolve host: master4.ddnet.tw
[2021-05-28 12:08:57][http]: http https://master3.ddnet.tw/ddnet/15/servers.json
[2021-05-28 12:08:57][http]: task failed. libcurl error: Could not resolve host: master3.ddnet.tw
[2021-05-28 12:08:57][http]: http https://master1.ddnet.tw/ddnet/15/servers.json
[2021-05-28 12:08:57][http]: task done https://master1.ddnet.tw/ddnet/15/servers.json
[2021-05-28 12:08:57][http]: http https://master1.ddnet.tw/ddnet/15/servers.json
[2021-05-28 12:08:58][http]: task done https://master1.ddnet.tw/ddnet/15/servers.json
[2021-05-28 12:08:58][serverbrowse_http]: found master, url='https://master1.ddnet.tw/ddnet/15/servers.json' time=201ms
[2021-05-28 12:08:58][serverbrowse_http]: determined best master, url='https://master1.ddnet.tw/ddnet/15/servers.json' time=201ms
[2021-05-28 12:08:58][http]: http https://master1.ddnet.tw/ddnet/15/servers.json
[2021-05-28 12:08:58][http]: task done https://master1.ddnet.tw/ddnet/15/servers.json

Now:

[2021-05-28 12:29:58][http]: http https://master1.ddnet.tw/ddnet/15/servers.json
[2021-05-28 12:29:59][http]: task done https://master1.ddnet.tw/ddnet/15/servers.json
[2021-05-28 12:30:00][serverbrowse_http]: found master, url='https://master2.ddnet.tw/ddnet/15/servers.json' time=799ms
[2021-05-28 12:30:00][serverbrowse_http]: found master, url='https://master1.ddnet.tw/ddnet/15/servers.json' time=43ms
[2021-05-28 12:30:00][serverbrowse_http]: determined best master, url='https://master1.ddnet.tw/ddnet/15/servers.json' time=43ms
2021-05-28 12:30:27 +02:00
def badaae58e9 [http]: http -> [http]: fetching 2021-05-28 12:11:59 +02:00
heinrich5991 436e202cbd Add client-side HTTP server info
Summary
=======

The idea of this is that clients will not have to ping each server for
server infos which takes long, leaks the client's IP address even to
servers the user does not join and is a DoS vector of the game servers
for attackers.

For the Internet, DDNet and KoG tab, the server list is entirely fetched
from the master server, filtering out servers that don't belong into the
list.

The favorites tab is also supposed to work that way, except for servers
that are marked as "also ping this server if it's not in the master
server list".

The LAN tab continues to broadcast the server info packet to find
servers in the LAN.

How does it work?
=================

The client ships with a list of master server list URLs. On first start,
the client checks which of these work and selects the fastest one.
Querying the server list is a HTTP GET request on that URL. The
response is a JSON document that contains server infos, server addresses
as URLs and an approximate location.

It can also contain a legacy server list which is a list of bare IP
addresses similar to the functionality the old master servers provided
via UDP. This allows us to backtrack on the larger update if it won't
work out.

Lost functionality
==================

(also known as user-visible changes)

Since the client doesn't ping each server in the list anymore, it has no
way of knowing its latency to the servers.

This is alleviated a bit by providing an approximate location for each
server (continent) so the client only has to know its own location for
approximating pings.
2021-05-14 01:00:39 +02:00
heinrich5991 d6097cb64c Automatically use HTTPS compression if the server supports it
Our typical HTTPS answer is reduced by 85% by this. Our JSON seems to
compress really well.
2021-03-07 22:41:34 +01:00
def 199da2410d Prevent OpenSSL SIGPIPE (fixes #3513)
by ignoring the signal ourselves

See https://curl.se/libcurl/c/CURLOPT_NOSIGNAL.html
2021-01-22 18:28:15 +01:00
def e592afc98c Add Thread Safety Analysis
and annotate some code: https://clang.llvm.org/docs/ThreadSafetyAnalysis.html
2020-12-23 18:13:21 +01:00
def cd00c4c04b Move PNG loading in SkinDL to fetch thread (fixes #3398)
Sorry for the trouble, seems to work.
2020-12-13 00:08:46 +01:00
def 165857a5a8 Fix variable names manually 2020-11-02 22:40:40 +01:00
def b1f0fd8969 Enable modernize-loop-convert clang-tidy check
and run clang-format afterwards

https://clang.llvm.org/extra/clang-tidy/checks/modernize-loop-convert.html
2020-11-02 22:40:24 +01:00
def 040876b0d3 Log close error to console 2020-10-15 17:48:22 +02:00
def 3be8a592e5 Run clang-format
Purely automatic change. In case of conflict with this change, apply the
other change and rerun the formatting to restore it:

$ python scripts/fix_style.py
2020-09-26 21:50:15 +02:00
def 2be54a30ad HTTP: Set final m_State after running completion function
Ensures that we don't delete a file that is being used by ingame map
downloader already.

As reported by Jupeyy in #2901
2020-09-24 10:21:12 +02:00
bors[bot] fddb8b34c0
Merge #2898 #2901 #2908
2898: Inform players that they are in team already r=heinrich5991 a=def-



2901: Remove failed downloaded files immediately r=heinrich5991 a=def-

Especially skins were only deleted after they were requested again

2908: Don't log skin downloading progress r=heinrich5991 a=def-

since it's not really interesting for most players, as long as things work fine

Co-authored-by: def <dennis@felsin9.de>
2020-09-22 14:57:20 +00:00
def 56b152e962 Remove failed downloaded files immediately
Especially skins were only deleted after they were requested again
2020-09-22 16:51:40 +02:00
def 1505b64af0 Don't log skin downloading progress
since it's not really interesting for most players, as long as things work fine
2020-09-22 16:36:01 +02:00
def f4d763c895 Init curl first, then call curl_version_info 2020-09-19 15:10:32 +02:00
def b7c3a6dfd8 Allow different timeouts for different downloads 2020-09-04 19:58:32 +02:00
def c458ee52b0 Remove need for custom certificate
in anticipation of using CloudFlare
2020-08-19 17:54:05 +02:00
def 539c20a200 Add support for PLATFORM_SUFFIX define
So that we can set it like this:

  CXXFLAGS='-DPLATFORM_SUFFIX="steam"' CPPFLAGS='-DPLATFORM_SUFFIX="steam"' make
2020-08-19 13:41:06 +02:00
def cf77b567f8 Allow any update.json size
just grew over 16 KB and thus updates failed. I still need to setup
update5 server and will need to release a last release on update4 to
have a seemless upgrade-path.
2020-06-25 00:11:00 +02:00
Learath 6919ed3041 Use updated stub, switch around versions 2020-03-23 15:46:57 +01:00
Learath 060d60f2ea Output library versions 2020-03-23 15:46:57 +01:00
Dennis Felsing cde07b420b Implement changes suggested by -Wuseless-cast
But don't enable it yet because I'm not sure what the best way is.
2019-04-11 19:54:43 +02:00
def e539bbacb1 Handle FS failures while updating (fixes #1560)
Not sure if I caught all locations
2019-04-06 19:24:40 +02:00
def bc91659aa8 Get GAME_RELEASE_VERSION back, fix compatibility filter in server browser 2019-03-06 21:02:06 +01:00
def deb2494692 Don't suggest to downgrade, clean up versioning (fixes #1440) 2019-03-03 20:31:23 +01:00
Dennis Felsing 5c18d6c481 Remove /modhelp (fixes #1401) 2018-12-12 10:05:44 +01:00
Renamed from src/engine/shared/http.cpp (Browse further)