Commit graph

366 commits

Author SHA1 Message Date
heinrich5991 9b4371f18f Change log format
Remove square brackets to reduce the amount of space used.

Add log level indicator. The position between timestamp and system was
chosen because it is at a fixed position (unlike after the system) but
the log still remains naively sortable (which wouldn't happen if we were
to place it in front of the timestamp.

Before:
```
[2022-04-29 15:25:37][engine]: running on unix-linux-amd64
[2022-04-29 15:25:37][engine]: arch is little endian
[2022-04-29 15:25:37][storage]: added path '$USERDIR' ('/path/to/home/.teeworlds')
[2022-04-29 15:25:37][storage]: added path '$DATADIR' ('data')
[2022-04-29 15:25:37][storage]: added path '$CURRENTDIR' ('/path/to/ddnet')
[2022-04-29 15:25:37][host_lookup]: host='localhost' port=0 1
[2022-04-29 15:25:37][host_lookup]: host='localhost' port=0 2
[2022-04-29 15:25:37][console]: executing 'autoexec_server.cfg'
```

After:
```
2022-04-29 15:25:37 I engine: running on unix-linux-amd64
2022-04-29 15:25:37 I engine: arch is little endian
2022-04-29 15:25:37 I storage: added path '$USERDIR' ('/path/to/home/.teeworlds')
2022-04-29 15:25:37 I storage: added path '$DATADIR' ('data')
2022-04-29 15:25:37 I storage: added path '$CURRENTDIR' ('/path/to/ddnet')
2022-04-29 15:25:37 I host_lookup: host='localhost' port=0 1
2022-04-29 15:25:37 I host_lookup: host='localhost' port=0 2
2022-04-29 15:25:37 I console: executing 'autoexec_server.cfg'
```
2022-09-14 19:37:55 +02:00
Robert Müller 0bde042c11 Always print warnings/failures in integration test
This makes it easier to debug issues when they appear in the CI.
2022-08-28 16:22:16 +02:00
Robert Müller 6bd1e96aa7 Add demo recorder/player to integration test
- record and playback client demo
- record and playback server demo
2022-08-26 20:58:08 +02:00
Dennis Felsing 06a971bcd1 Fix rest of pylint manually 2022-07-27 09:37:25 +02:00
Dennis Felsing 9f8e97fbeb Run flynt -a 2022-07-27 09:37:24 +02:00
Dennis Felsing cfdb4646d1 Update scripts/check_identifiers.py
Co-authored-by: heinrich5991 <heinrich5991@gmail.com>
2022-07-27 09:37:24 +02:00
def 7379a64004 [WIP] Require Python 3.6 (f-strings) and fix pylints
So far only done scripts directory, will do the rest if this is
considered good
2022-07-27 09:37:24 +02:00
ChillerDragon f238c55013 Fix integration tests hanging on crashes
When the client or server crash the fifo will be dead
then the cleanup command hangs at

	echo "quit" > client1.fifo

because the fifo is not responding and thus it is stuck there.
which causes the user to probably ctrl+c at some point
which then avoids client2 to be quit which might not have crashed

So this commit checks if a fail_client1.txt file exits and if it crashed
already it will skip the call to the broken fifo
2022-07-13 17:03:41 +02:00
ChillerDragon 28c4a8a346 Sort CI logs before diff to fix race condition
Logging to file and to stdout might order differently
due to multiple threads
2022-06-28 16:26:40 +02:00
def b1d1bb6739 Don't register integration test servers
Also print out more of the diff
2022-06-26 08:59:44 +02:00
bors[bot] 937a9cc0e2
Merge #5495
5495: Fix Android build r=heinrich5991 a=Jupeyy

- Video recorder off
- small fix in logger.cpp

Tested on Java-17

## Checklist

- [ ] Tested the change ingame
- [ ] Provided screenshots if it is a visual change
- [ ] Tested in combination with possibly related configuration options
- [ ] Written a unit test if it works standalone, system.c especially
- [ ] Considered possible null pointers and out of bounds array indexing
- [ ] Changed no physics that affect existing maps
- [ ] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssanitizer--undefinedbehavioursanitizer-or-valgrinds-memcheck) (optional)


Co-authored-by: Jupeyy <jupjopjap@gmail.com>
2022-06-25 08:54:11 +00:00
Jupeyy a22a8dce1c Fix Android build 2022-06-25 10:43:24 +02:00
Dennis Felsing 0081e99fc7 integration test: print log diff for easier result check 2022-06-25 01:39:59 +02:00
ChillerDragon c951a3cb89 Add logfiles and map changes to integration test
Cover map change
Cover logfiles
Ensure client1 always connects first
2022-06-23 15:52:28 +02:00
bors[bot] b75289df35
Merge #4829 #5343 #5465
4829: Added map_replace_area tool r=def- a=sctt

Motivation:
sometimes mappers need to copy a certain area from a map to another (but also to a different position of the same map).
that's very time consuming on complex maps, as tiles and quads layers have to be manually copied one by one.

map_replace_area is designed to automatically replace all the tiles and quads from a specific area to another.
`Usage: map_replace_area <from_map> <from_x> <from_y> <to_map> <to_x> <to_y> <width> <height> <output_map>`

PoW screenshots attached below.

note: for the moment map_replace_area suppose that you are working on the same map  (that's what i needed), that means source and destination maps must have the same layers structure, or else an error is returned.
but i understand it might be useful to also work on totally different maps, in that case users must be able specify layers they want to consider by associating source map layers to destination map layers (by using a config file maybe).
i'm probably going to make another PR to add this enhancement in the future.

PoW:
![Screenshot from 2022-03-16 18-08-12](https://user-images.githubusercontent.com/3328841/158650515-17c31639-28f7-4e19-954a-b5734ee82703.png)
![Screenshot from 2022-03-16 18-08-43](https://user-images.githubusercontent.com/3328841/158650512-6e21f2b4-538f-4974-aaa2-2983551d24b4.png)
![Screenshot from 2022-03-16 18-11-50](https://user-images.githubusercontent.com/3328841/158650510-d00a05a9-a2e8-4df2-8674-0c80fd894f66.png)

5343: Add rcon and chat to integration test r=def- a=ChillerDragon

Add more action to the integration test script: chat messages, chat commands, rcon commands.
This unlocks more code coverage at runtime to catch asan issues.

It adds a very restrictive regex on the log format. But I am happy to maintain that in case changes to the log output happen.
The current version is already supporting heinrichs refactor https://github.com/ddnet/ddnet/pull/5036

Also ensure chat messages arrive in the correct format. Would have catched the following issues:
https://github.com/ddnet/ddnet/issues/5342
https://github.com/ddnet/ddnet/issues/5340
https://github.com/ddnet/ddnet/issues/5302
https://github.com/ddnet/ddnet/pull/5126

DEPENDS ON:

https://github.com/ddnet/ddnet/issues/5342
https://github.com/ddnet/ddnet/issues/5340

5465: Change from pnglite to libpng for PNG reading r=def- a=heinrich5991

This is desirable mainly because libpng is maintained and pnglite is
not. pnglite was last updated in 2007 (15 years ago) and probably has a
lot of security vulnerabilities.

libpng is an actively maintained library also used by browsers like
Firefox or Chromium, so it's less likely to contain security
vulnerabilities, also it's more likely to be packaged by Linux
distributions.

This also refuses to load images of types not supported by pnglite,
which allows us to think about backward compatibility while also
introducing libpng.

## Checklist

- [x] Tested the change ingame
- [ ] Provided screenshots if it is a visual change
- [ ] Tested in combination with possibly related configuration options
- [ ] Written a unit test if it works standalone, system.c especially
- [ ] Considered possible null pointers and out of bounds array indexing
- [ ] Changed no physics that affect existing maps
- [ ] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssanitizer--undefinedbehavioursanitizer-or-valgrinds-memcheck) (optional)


Co-authored-by: sctt <scottistefano91@gmail.com>
Co-authored-by: term <term@term.sinervis.pri>
Co-authored-by: f <scottistefano91@gmail.com>
Co-authored-by: heinrich5991 <heinrich5991@gmail.com>
Co-authored-by: ChillerDragon <ChillerDragon@gmail.com>
Co-authored-by: def <dennis@felsin9.de>
Co-authored-by: Jupeyy <jupjopjap@gmail.com>
2022-06-21 16:21:47 +00:00
ChillerDragon bc243952bd Increase client boot timeout for integration test 2022-06-21 01:15:06 +02:00
ChillerDragon afd4504f96 Fix integration argument parsing 2022-06-17 23:14:13 +02:00
Robert Müller 8757b1f41e Add joystick support to engine 2022-06-13 20:40:36 +02:00
def 23af9ef448 Tell what has to be changed in check_header_guards.py
Example:

Wrong header guard in src/game/bezier.h, is: #ifndef GAAME_BEZIER_H, should be: #ifndef GAME_BEZIER_H
2022-06-12 12:34:21 +02:00
def 5768e52a13 Minor fixes 2022-06-07 00:33:01 +02:00
Robert Müller f500ced16e Port scripts/gen_keys.py to Python 3, add missing NOLINT comment 2022-06-06 22:14:05 +02:00
def 442ae7dc13 Make it easier to run integration tests
Entire worflow is now:

mkdir san
cd san
CC=clang CXX=clang++ CXXFLAGS="-fsanitize=address,undefined -fsanitize-recover=address,undefined -fno-omit-frame-pointer" CFLAGS="-fsanitize=address,undefined -fsanitize-recover=address,undefined -fno-omit-frame-pointer" cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo -GNinja ..
ninja run_integration_tests
2022-06-06 18:53:59 +02:00
def 9a89e58a16 Add lsan.supp for opengl leak (fixes #5296)
See also Jupeyy's explanation in
https://github.com/ddnet/ddnet/issues/4294 for why we want to ignore
this:

> sadly components have no OnDestruct on anything like that
> so aslong it doesn't actually leak it's just non destructed memory

So it's something we allocate once for the entire client lifetime from
my understanding.

=================================================================
==421461==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 411760 byte(s) in 10 object(s) allocated from:
    #0 0x55eaf0c08a29 in malloc (/media/ddnet/integration_test/DDNet+0x736a29)
    #1 0x55eaf0de9f72 in CCommandProcessorFragment_OpenGL2::Cmd_CreateBufferObject(CCommandBuffer::SCommand_CreateBufferObject const*) /media/ddnet/src/engine/client/backend/opengl/backend_opengl.cpp:1898:25
    #2 0x55eaf0dd82f5 in CCommandProcessorFragment_OpenGL::RunCommand(CCommandBuffer::SCommand const*) /media/ddnet/src/engine/client/backend/opengl/backend_opengl.cpp
    #3 0x55eaf0f73229 in CCommandProcessor_SDL_GL::RunBuffer(CCommandBuffer*) /media/ddnet/src/engine/client/backend_sdl.cpp:246:20
    #4 0x55eaf0f70568 in CGraphicsBackend_Threaded::ThreadFunc(void*) /media/ddnet/src/engine/client/backend_sdl.cpp:75:25
    #5 0x55eaf0d679d7 in thread_run(void*) /media/ddnet/src/base/system.cpp:686:2
    #6 0x7f08b788c54c  (/usr/lib/libc.so.6+0x8c54c)

SUMMARY: AddressSanitizer: 411760 byte(s) leaked in 10 allocation(s).
2022-06-05 20:01:02 +02:00
ChillerDragon 44a79d5783 Add chat and rcon to integration test 2022-06-05 19:26:39 +02:00
ChillerDragon eebabaaf82 Fix indent 2022-06-05 19:26:22 +02:00
heinrich5991 b0fcda57c2 Fix integration test waiting for client to close
As described in #5295, the integration test script waits for the client
to close before launching the next one. I don't know what causes this,
but disabling full-screen mode works around the issue.

I don't know if this is okay. @ChillerDragon?

Fixes #5295.
2022-06-02 01:47:10 +02:00
def b3d8e05e00 Add valgrind to integration test 2022-05-26 23:52:29 +02:00
Dennis Felsing e3116217de test -> integration_test 2022-05-26 01:34:18 +02:00
Dennis Felsing 68bcd21eff asan+ubsan clean 2022-05-26 01:22:14 +02:00
ChillerDrgon 05efc25f32 Add integration test script and coverage map 2022-05-26 00:37:45 +02:00
Dennis Felsing 1158bbfbc5 Don't assume script name/path 2022-05-23 17:10:19 +02:00
Chairn 76e6d0b120 Fix pylint error 2022-05-07 01:21:58 +02:00
Chairn 0cd61e6ae1 Added header guard exception 2022-05-07 01:08:51 +02:00
Chairn 5925181acc Updated unicode script generation 2022-05-06 20:31:24 +02:00
def cd87fc1ef7 Fix move_sqlite (fixes #4902) 2022-03-28 00:35:39 +02:00
Jupeyy bb1ed45c3d Emscripten support 2022-03-25 12:42:18 +01:00
Jupeyy e02ebca2c9 Rename lib generation scripts 2022-03-25 11:46:32 +01:00
Jupeyy fbeba9eb5f Add Vulkan backend 2022-03-21 12:39:51 +01:00
Dennis Felsing af36070c91 Update ddnet-libs, fix shellcheck 2022-02-24 10:51:31 +01:00
Jupeyy 8bed1c7267 Add module offset 2022-02-24 10:45:58 +01:00
heinrich5991 1820a0e168 Add some checksum functions to the client
Let's see if it works out, if not, we can revert it.
2022-01-31 21:31:26 +01:00
Jupeyy 55a1406641 Further android build fixes 2022-01-20 11:04:47 +01:00
Jupeyy 4bcddc8dae fix android curl cmake flags 2022-01-19 10:44:48 +01:00
def fec20679f1 Create custom look for DMG using dmgbuild
(nice background image still missing)
2021-12-26 00:24:55 +01:00
def 052502a3d1 Fix local shellcheck warnings in android/files/build.sh
In ./scripts/android/files/build.sh line 3:
[ x"$1" == x ] && {
  ^---^ SC2268 (style): Avoid x-prefix in comparisons as it no longer serves a purpose.

Did you mean:
[ "$1" == "" ] && {
2021-12-21 17:52:06 +01:00
def 28ac9396cc Support libfat directory for macOS, use arm64 libs (fixes #3250)
Builds and runs successfully with:
  cmake -DCMAKE_BUILD_TYPE=Release -GNinja -DPREFER_BUNDLED_LIBS=ON -DCMAKE_OSX_ARCHITECTURES="arm64;x86_64"
  ninja package_dmg

Tested successfully with native arm64 and rosetta

No discord support for universal binary
2021-12-20 00:17:38 +01:00
Jupeyy b216b6744a Add Android build scripts 2021-08-24 12:18:20 +02:00
heinrich5991 1f68e11445 Mark the inverted exclamation mark as confusable with i
They look pretty much the same in our font, but for some reason, the
inverted exclamation mark doesn't have confusables.
2021-07-27 02:43:04 +02:00
heinrich5991 707e13ff7a Update confusables_data.h to Unicode 13.0.0 2021-07-27 02:41:55 +02:00
def f867d9c2ac Only use correct clang-format version 2021-04-17 20:24:57 +02:00
def 8fc1fe3811 Retry hdiutil create if it fails (fixes #3711) 2021-03-16 17:56:36 +01:00
def 387b8ef51d Fix dmg.py
Traceback (most recent call last):
  File "/home/deen/isos/ddnet/ddnet-source/scripts/dmg.py", line 106, in <module>
    main()
  File "/home/deen/isos/ddnet/ddnet-source/scripts/dmg.py", line 103, in main
    dmg.create(volume_name=args.volume_name, directory=args.directory, dmg=args.output, symlinks=symlinks)
  File "/home/deen/isos/ddnet/ddnet-source/scripts/dmg.py", line 59, in create
    self._create_hfs(hfs, volume_name, output_size)
  File "/home/deen/isos/ddnet/ddnet-source/scripts/dmg.py", line 42, in _create_hfs
    with open(hfs, 'wb') as f:
TypeError: expected str, bytes or os.PathLike object, not tuple
2021-03-04 10:01:22 +01:00
Edgar 6e76c46c4c
Fix Insecure temporary file 2021-03-01 09:14:25 +01:00
def 13a8dfa307 Add pylint
and fix occurences
2020-12-09 10:40:28 +01:00
def 1dcc1e4026 Run shellcheck over check_dilate.sh 2020-12-08 15:02:16 +01:00
def 0822f9cecd Add check_dilate 2020-11-24 11:15:24 +01:00
def 5b24e95b29 Check translations in github, check formatting strings, fix 2020-11-12 08:57:20 +01:00
heinrich5991 091db3df32 Add scripts to detect variable name style violations
Currently very user-unfriendly, call like this:

```
env CXXFLAGS="-I<builddir>/src -I<includedir_lib1> -I<includedir_lib2> …" python scripts/extract_identifiers.py src/game/**.cpp src/engine/client/**.cpp src/engine/server/**.cpp src/engine/shared/**.cpp > identifiers
python scripts/check_identifiers.py < identifiers
```
2020-11-08 17:13:13 +01:00
def 06b534c07d Create fallback wordlist in sourcecode (fixes #3206) 2020-11-03 17:01:16 +01:00
heinrich5991 8427e99672 Don't run the style fixer for generated files
Fixes #2962.
2020-10-02 19:30:05 +02:00
def 2e8e19290e fix_style.py: Ignore generated key headers 2020-09-26 21:50:13 +02:00
heinrich5991 678b2b15c3 Change fix_style.py script to unconditionally format everything 2020-09-26 21:41:24 +02:00
def 2d8a1ee2b7 Make fix_style.py look at files, not lines 2020-09-26 21:41:20 +02:00
def 3daa798639 Implement script/languages support for contexts 2020-09-15 10:30:49 +02:00
heinrich5991 b235637713 Find the common ancestor for the style checking
This allows the style checker to only look at the changes done in the
current branch, and not the other commits that were done in the base
branch.
2020-09-11 00:57:03 +02:00
def ecdf648930 Make fix_style executable 2020-09-10 22:58:30 +02:00
heinrich5991 df46ff8557 Add a tools to fix the style of new pull requests 2020-09-10 04:11:58 +02:00
def 63f6ecd19d Ignore duplicate records when inserting (fixes #2636) 2020-08-29 08:26:03 +02:00
Zwelf ca24611c0c Do not recommend to deleting ddnet-server.sqlite database
as it might be still in use or there might be even new ranks added to the database
2020-08-12 23:38:09 +02:00
Zwelf 9848b3c759 SQLite transfer script: Warn if database does not exist yet and handle partially created sqlite tables 2020-08-12 23:38:09 +02:00
Zwelf 7834c2c1d8 Add file score import script by heinrich5991 2020-08-03 12:58:52 +02:00
Zwelf e483011df6 Add script to move SQLite ranks to MySQL 2020-08-03 12:58:52 +02:00
def 59cf78be24 Always recreate the language files in same order
Alphabetically as the files are located in directory structure.

Keeps the language files more stable
2020-06-30 21:38:14 +02:00
def 2d581e0612 copy_fix: Don't generate comment line, just add missing translations at the end 2020-06-26 23:28:44 +02:00
def 2d6e56fec8 Translation fixes
- Code didn't work with newlines, get rid of them
- Some stylistic improvements
2020-06-26 18:34:50 +02:00
def f925822c5e Add instructions for translation stats 2020-06-25 22:12:35 +02:00
Learath 09fad44b87 Strip rpath before adding the correct one 2020-04-30 15:04:51 +02:00
def 31ae4c4d5a use pid in file names of temporary files
to prevent race conditions with multiple clients running when saving
maps, config on quit, during upgrade
2020-02-12 23:14:30 +01:00
heinrich5991 96069f42a9 Add a script to generate a stub libcurl library
Linking to this library will make DDNet compatible with all distros that
ship libcurl.so.4 with arbitrary version data attached. Distros
sometimes add versioning information to the symbols exported by curl
that artificially (because they're compatible with each other) limit
where our releases can be executed if dynamically linked.

This should reduce release size a bit and let's us take advantage of
automatically updated libraries.
2020-01-05 17:58:52 +01:00
Dennis Felsing 0498df2508 Update localizations (Thanks to Cellegen for report)
- Remove duplicate laser entries
- Update menus_settings.cpp comment for scripts
- Update instructions for scripts
- Run script to get updated localizations
2019-12-02 09:09:01 +01:00
def 94f3d97ddf Remove unmaintained Android code (fixes #1575) 2019-04-10 19:40:50 +02:00
def faa3cc195d UTF8 nocase compare & use for chat TAB completion
- As suggested by Arseniy Zarche
- Also updated confusables to Unicode 12
2019-01-09 08:31:03 +01:00
12pm d46a5c7994 Fix output for usernames with double quotes and semicolons 2018-11-15 18:30:44 +01:00
12pm 59334a7068 Fix output for usernames with spaces 2018-11-10 13:52:13 +01:00
heinrich5991 41d0f86b12 Use __file__ instead of sys.argv[0]
Thanks to @bojidar-bg for the hint.
2018-07-25 09:17:10 +02:00
heinrich5991 945ea1cf28 Make language scripts callable from anywhere 2018-07-25 09:17:09 +02:00
heinrich5991 3a8cc43e13 Move language scripts to scripts/ 2018-07-25 00:34:02 +02:00
yangfl 81a39c229b Fix typo 2018-07-10 17:29:02 +08:00
heinrich5991 925aff21d4 Fix all the header guards and adjust the script a little 2018-07-06 16:11:38 +02:00
heinrich5991 d7f30c574f Update scripts/check_header_guards.py
Python 3 compatibility, return code, changing directory before script
starts.
2018-07-06 15:45:05 +02:00
heinrich5991 f46f811280 Remove a bunch of unused scripts
scripts/compiler.py: Looks like an older version of datasrc/datatypes.py
scripts/copyright.py: Checks copyright headers in files
scripts/count_source.sh: Counts source files, from SVN(!) times
scripts/dat2c: Writes a C constant with the contents of a file
scripts/font_converter.py: Conversion tool for an old font format
scripts/font_installer.sh: Calls the conversion tool on all fonts
scripts/linecount.sh: Counts source code lines (using `wc` and `find`)
scripts/make_docs.sh: Generates source code docs that haven't existed for a long time
scripts/mass_server.py: Starts a lot of `fake_server`
scripts/netobj.py: Looks like an older version of datasrc/datatypes.py
scripts/nicks.txt: Auxiliary file for scripts/mass_server.py
scripts/png.py: Conversion of image file formats
scripts/process_blame.py: SVN-era source count
2018-07-06 15:14:13 +02:00
heinrich5991 6d88a29910 Add LATIN SMALL LETTER L confusable
This makes `LATIN SMALL LETTER L` confusable with `LATIN SMALL LETTER I`
because `CYRILLIC SMALL LETTER PALOCHKA` "ӏ" (mapping to i) wasn't
confusable with `LATIN SMALL LETTER L` "l" (mapping to l) before.

Also add tests for `str_utf8_comp_confusable`.
2018-03-20 21:11:44 +01:00
heinrich5991 565f963a7b Properly hide stderr in scripts/git_revision.py 2018-03-02 03:31:45 +01:00
heinrich5991 b285d777f3 Python 2 support for scripts/git_revision.py on error /o\ 2018-03-02 03:22:44 +01:00
heinrich5991 1387ee9a3a Regenerate version file on git changes
Previously, this file had to be manually deleted if you wanted to have a
new version compiled in.
2018-03-01 16:50:29 +01:00
yangfl 1620fc9b0f Fix typo 2018-02-07 16:10:40 +08:00
Dennis Felsing efe053033b Remove make_src.py 2018-01-29 14:38:42 +01:00
Learath2 050a34295f Remove make_release, scrub references to lua files 2018-01-29 13:37:26 +01:00
Learath2 b26eeb235d Scrub more references to bam 2018-01-28 22:47:03 +01:00
bors[bot] a2f010c1fe Merge #941
941: Add cross-compiled DMG archives r=Learath2 a=heinrich5991

Currently they only contain the client and the archives are a lot larger
than the current release artifacts.
2017-12-08 14:20:49 +00:00