By adding `CDataFileWriter::ECompressionLevel` to replace usage of zlib internal compression levels in the `CDataFileWriter` API.
Use `std::numeric_limits<int>::max()` instead of `INT_MAX` in one case where the latter was only declared by the transitive zlib include. The `limits` header is already included and its use is more fitting for C++ code.
Adapt the `CDataFileReader::GetItem` function so it optionally also returns a `CUuid` for UUID-based map items, including for items with unknown UUIDs where the item type will be `-1`. Adapt the `CDataFileWriter::AddItem` function so it optionally also accepts a `CUuid` which will be used if the item type is `-1`.
The additional checks for invalid map item types in the map tools are removed again and instead the new UUID parameters are used so map items with unknown UUIDs are written back to maps correctly.
Closes#7701.
Previously, the client would crash with an assertion when writing more than 1024 items, 1024 data items or 64 extended item types to a map file. This arbitrary limitation is removed by using `std::vector`s, so much larger maps can be written. This was only a limitation when writing map files. Old clients can correctly read the map files having more items/data but will crash when the map is saved in the editor.
Add assertions to prevent map items and data that are too large to be represented by the map format from being written to maps. Additionally, ensure that the size of the whole map file is not too large to be represented by the map header.
Prevent `malloc` of size 0, which is implementation defined and should be avoided, when adding items without data, which happens for example when adding an empty array of envelope points.
Simplify the usage of datafile reader and writer by adding utility functions to read and write zero-terminated UTF-8 strings.
Improve validation of string data read from datafiles. It is ensure that string data is null-terminated, has no internal NUL-characters and is valid UTF-8.
Fix loading of external sounds in the editor. The wrong path variable was being used, so the sound files would not be loaded from correct folder.
Add tests for new datafile reader/writer functions.
When `IMap::Load` fails, other components will continue to use the old map. However, if `IMap::Load` failed after the map was already successfully read with the datafile reader then other components kept their pointers to the old, invalid datafile reader items and data, causing random crashes in collision code. This is fixed by using a separate datafile reader to read the new map and only applying the datafile reader globally when loading was entirely successfully.
An error message is added for the case that a map has an unsupported version, which is currently the only case where a map can fail to load after the datafile was read successfully.
In particular, the block maps `blmapPepe`, `blmapV5` and `blmapDT-UPTU` did not have a version map item and were fixed separately.
Closes#7218. Regression from #5737.
Check if datafile data cannot be read entirely (according to the data size specified in the header) and check for decompression errors. In case of errors, let `GetData` return `nullptr` and `GetDataSize` return `0` for the respective index.
Internally the decompressed size is set to `-1` for data which failed to load, so loading of those data will not be attempted again because it would only fail again and can cause additional log messages.
Previously it reported the internal file data size (compressed). This
made the `map_resave` tool do the wrong job.
Co-authored-by: heinrich5991 <heinrich5991@gmail.com>
Compressing the data with zlib takes the majority of the time when saving a datafile. Therefore, compressing is now delayed until the `CDataFileWriter::Finish` function is called. This function is then off-loaded to another thread to make saving maps in the editor not block the rendering.
A message "Saving…" is shown in the bottom right of the editor view while a job to save a map is running in the background.
While a map is being finished in a background thread another save for the same filename cannot be initiated to prevent multiples accesses to the same file.
Closes#6762.
Purely automatic change. In case of conflict with this change, apply the
other change and rerun the formatting to restore it:
$ python scripts/fix_style.py
SHA256 was chosen because it is reasonably standard, the file names
don't explode in length (this rules out SHA512) and it is supported by
basically all versions of OpenSSL (this rules out SHA512/256 and SHA3).
The protocol is changed in a backward compatible way: The supporting
server sends the SHA256 corresponding to the map in the `MAP_DETAILS`
message prior to sending the `MAP_CHANGE` message. The client saves the
SHA256 obtained from the `MAP_DETAILS` message until the next
`MAP_CHANGE` message.
For servers not supporting this protocol, the client falls back to
simply opening maps like in the previous scheme.
Remove the `map_version` tool, it is not being used and would have been
a little bit effort to update.
Use the OpenSSL implementation of SHA256 if it is supported, otherwise
fall back to a public domain one.
Fix#1127.
ChillerDragon
