edg-l/ddnet
1
0
Fork 0
mirror of https://github.com/ddnet/ddnet.git synced 2024-11-18 22:18:19 +00:00
Code Issues Packages Projects Releases Wiki Activity
ddnet/src/test/str.cpp

1272 lines
40 KiB
C++
Raw Normal View History

Add possibility to ban players by name This uses the Unicode confusable data together with judging how close two strings are by using the Levenshtein distance. Adds the commands `name_ban`, `name_unban` and `name_bans`. Kicks players who join using a banned name and doesn't allow ingame players to change their names to the banned ones.
2018-03-06 17:41:18 +00:00
#include <gtest/gtest.h>
#include <base/system.h>
Add validation for `StrToInts` and `IntsToStr` Add strict validation for `StrToInts` function. Because this function should only be used with trusted internal strings, assertions are added to ensure that the string is not truncated. Some buffer sizes are adjusted accordingly, so truncation cannot happen. Add less strict validation for `IntsToStr` function. An additional argument specifying the size of the output buffer is added to assert that the size of the output buffer is sufficient. However, because this function is used to decode data sent by the server and read from maps and ghosts, invalid input data should never result in crashes or invalid UTF-8 strings. The function will now unpack an empty string and return `false`, if the string contains invalid UTF-8. The inline definition of the functions is not wanted, because it requires adding a `system.h` include in `gamecore.h`. Therefore the tools now have to depend on game-shared, which previously only worked because the functions were inline. Tests are added to ensure the function still behaves the same as before for valid inputs and correctly handles invalid inputs.
2024-03-04 17:05:12 +00:00
#include <game/gamecore.h>
Add possibility to ban players by name This uses the Unicode confusable data together with judging how close two strings are by using the Levenshtein distance. Adds the commands `name_ban`, `name_unban` and `name_bans`. Kicks players who join using a banned name and doesn't allow ingame players to change their names to the banned ones.
2018-03-06 17:41:18 +00:00
TEST(Str, Dist)
{
EXPECT_EQ(str_utf8_dist("aaa", "aaa"), 0);
EXPECT_EQ(str_utf8_dist("123", "123"), 0);
EXPECT_EQ(str_utf8_dist("", ""), 0);
EXPECT_EQ(str_utf8_dist("a", "b"), 1);
EXPECT_EQ(str_utf8_dist("", "aaa"), 3);
EXPECT_EQ(str_utf8_dist("123", ""), 3);
EXPECT_EQ(str_utf8_dist("ä", ""), 1);
EXPECT_EQ(str_utf8_dist("Hëllö", "Hello"), 2);
// https://en.wikipedia.org/w/index.php?title=Levenshtein_distance&oldid=828480025#Example
EXPECT_EQ(str_utf8_dist("kitten", "sitting"), 3);
EXPECT_EQ(str_utf8_dist("flaw", "lawn"), 2);
EXPECT_EQ(str_utf8_dist("saturday", "sunday"), 3);
}
Fix some system.c insanity `str_utf8_isspace` now returns true if the passed code point renders as a space, instead of when not. Add `str_utf8_trim_right`, use this function and `str_utf8_skip_whitespaces` in the server. Add tests for the three functions
2018-03-14 01:27:15 +00:00
TEST(Str, Utf8Isspace)
{
Add string and nameban tests with Unicode characters
2018-03-14 02:04:41 +00:00
EXPECT_TRUE(str_utf8_isspace(0x200b)); // Zero-width space
Fix some system.c insanity `str_utf8_isspace` now returns true if the passed code point renders as a space, instead of when not. Add `str_utf8_trim_right`, use this function and `str_utf8_skip_whitespaces` in the server. Add tests for the three functions
2018-03-14 01:27:15 +00:00
EXPECT_TRUE(str_utf8_isspace(' '));
EXPECT_FALSE(str_utf8_isspace('a'));
Add string and nameban tests with Unicode characters
2018-03-14 02:04:41 +00:00
// Control characters.
for(char c = 0; c < 0x20; c++)
{
EXPECT_TRUE(str_utf8_isspace(c));
}
Fix some system.c insanity `str_utf8_isspace` now returns true if the passed code point renders as a space, instead of when not. Add `str_utf8_trim_right`, use this function and `str_utf8_skip_whitespaces` in the server. Add tests for the three functions
2018-03-14 01:27:15 +00:00
}
TEST(Str, Utf8SkipWhitespaces)
{
EXPECT_STREQ(str_utf8_skip_whitespaces("abc"), "abc");
EXPECT_STREQ(str_utf8_skip_whitespaces("abc "), "abc ");
EXPECT_STREQ(str_utf8_skip_whitespaces(" abc"), "abc");
Add string and nameban tests with Unicode characters
2018-03-14 02:04:41 +00:00
EXPECT_STREQ(str_utf8_skip_whitespaces("\xe2\x80\x8b abc"), "abc");
Fix some system.c insanity `str_utf8_isspace` now returns true if the passed code point renders as a space, instead of when not. Add `str_utf8_trim_right`, use this function and `str_utf8_skip_whitespaces` in the server. Add tests for the three functions
2018-03-14 01:27:15 +00:00
}
TEST(Str, Utf8TrimRight)
{
Run clang-format Purely automatic change. In case of conflict with this change, apply the other change and rerun the formatting to restore it: $ python scripts/fix_style.py
2020-09-26 19:41:58 +00:00
char A1[] = "abc";
str_utf8_trim_right(A1);
EXPECT_STREQ(A1, "abc");
char A2[] = " abc";
str_utf8_trim_right(A2);
EXPECT_STREQ(A2, " abc");
char A3[] = "abc ";
str_utf8_trim_right(A3);
EXPECT_STREQ(A3, "abc");
char A4[] = "abc \xe2\x80\x8b";
str_utf8_trim_right(A4);
EXPECT_STREQ(A4, "abc");
Fix some system.c insanity `str_utf8_isspace` now returns true if the passed code point renders as a space, instead of when not. Add `str_utf8_trim_right`, use this function and `str_utf8_skip_whitespaces` in the server. Add tests for the three functions
2018-03-14 01:27:15 +00:00
}
Add `LATIN SMALL LETTER L` confusable This makes `LATIN SMALL LETTER L` confusable with `LATIN SMALL LETTER I` because `CYRILLIC SMALL LETTER PALOCHKA` "ӏ" (mapping to i) wasn't confusable with `LATIN SMALL LETTER L` "l" (mapping to l) before. Also add tests for `str_utf8_comp_confusable`.
2018-03-20 20:11:44 +00:00
TEST(Str, Utf8CompConfusables)
{
EXPECT_TRUE(str_utf8_comp_confusable("abc", "abc") == 0);
EXPECT_TRUE(str_utf8_comp_confusable("rn", "m") == 0);
Add more tests for `str_utf8_comp_confusable` Also check first string being shorter than second string. And check strings with equal and unequal prefix and suffix.
2022-10-30 10:43:10 +00:00
EXPECT_TRUE(str_utf8_comp_confusable("m", "rn") == 0);
EXPECT_TRUE(str_utf8_comp_confusable("rna", "ma") == 0);
EXPECT_TRUE(str_utf8_comp_confusable("ma", "rna") == 0);
EXPECT_FALSE(str_utf8_comp_confusable("mA", "rna") == 0);
EXPECT_FALSE(str_utf8_comp_confusable("ma", "rnA") == 0);
EXPECT_TRUE(str_utf8_comp_confusable("arn", "am") == 0);
EXPECT_TRUE(str_utf8_comp_confusable("am", "arn") == 0);
EXPECT_FALSE(str_utf8_comp_confusable("Am", "arn") == 0);
EXPECT_FALSE(str_utf8_comp_confusable("am", "Arn") == 0);
Add `LATIN SMALL LETTER L` confusable This makes `LATIN SMALL LETTER L` confusable with `LATIN SMALL LETTER I` because `CYRILLIC SMALL LETTER PALOCHKA` "ӏ" (mapping to i) wasn't confusable with `LATIN SMALL LETTER L` "l" (mapping to l) before. Also add tests for `str_utf8_comp_confusable`.
2018-03-20 20:11:44 +00:00
EXPECT_TRUE(str_utf8_comp_confusable("l", "ӏ") == 0); // CYRILLIC SMALL LETTER PALOCHKA
Mark the inverted exclamation mark as confusable with i They look pretty much the same in our font, but for some reason, the inverted exclamation mark doesn't have confusables.
2021-07-27 00:43:04 +00:00
EXPECT_TRUE(str_utf8_comp_confusable("i", "¡") == 0); // INVERTED EXCLAMATION MARK
Add `LATIN SMALL LETTER L` confusable This makes `LATIN SMALL LETTER L` confusable with `LATIN SMALL LETTER I` because `CYRILLIC SMALL LETTER PALOCHKA` "ӏ" (mapping to i) wasn't confusable with `LATIN SMALL LETTER L` "l" (mapping to l) before. Also add tests for `str_utf8_comp_confusable`.
2018-03-20 20:11:44 +00:00
EXPECT_FALSE(str_utf8_comp_confusable("o", "x") == 0);
EXPECT_TRUE(str_utf8_comp_confusable("aceiou", "ąçęįǫų") == 0);
}
Add `str_startswith` and `str_endswith` Also add a couple of tests for both.
2018-07-25 08:23:07 +00:00
Add tests for `str_utf8_to_skeleton`
2022-11-01 22:32:58 +00:00
TEST(Str, Utf8ToSkeleton)
{
int aBuf[32];
EXPECT_EQ(str_utf8_to_skeleton("abc", aBuf, 0), 0);
EXPECT_EQ(str_utf8_to_skeleton("", aBuf, std::size(aBuf)), 0);
EXPECT_EQ(str_utf8_to_skeleton("abc", aBuf, std::size(aBuf)), 3);
EXPECT_EQ(aBuf[0], 'a');
EXPECT_EQ(aBuf[1], 'b');
EXPECT_EQ(aBuf[2], 'c');
EXPECT_EQ(str_utf8_to_skeleton("m", aBuf, std::size(aBuf)), 2);
EXPECT_EQ(aBuf[0], 'r');
EXPECT_EQ(aBuf[1], 'n');
EXPECT_EQ(str_utf8_to_skeleton("rn", aBuf, std::size(aBuf)), 2);
EXPECT_EQ(aBuf[0], 'r');
EXPECT_EQ(aBuf[1], 'n');
EXPECT_EQ(str_utf8_to_skeleton("ӏ", aBuf, std::size(aBuf)), 1); // CYRILLIC SMALL LETTER PALOCHKA
EXPECT_EQ(aBuf[0], 'i');
EXPECT_EQ(str_utf8_to_skeleton("¡", aBuf, std::size(aBuf)), 1); // INVERTED EXCLAMATION MARK
EXPECT_EQ(aBuf[0], 'i');
EXPECT_EQ(str_utf8_to_skeleton("ąçęįǫų", aBuf, std::size(aBuf)), 6);
EXPECT_EQ(aBuf[0], 'a');
EXPECT_EQ(aBuf[1], 'c');
EXPECT_EQ(aBuf[2], 'e');
EXPECT_EQ(aBuf[3], 'i');
EXPECT_EQ(aBuf[4], 'o');
EXPECT_EQ(aBuf[5], 'u');
}
UTF8 nocase compare & use for chat TAB completion - As suggested by Arseniy Zarche - Also updated confusables to Unicode 12
2019-01-07 22:49:20 +00:00
TEST(Str, Utf8ToLower)
{
EXPECT_TRUE(str_utf8_tolower('A') == 'a');
EXPECT_TRUE(str_utf8_tolower('z') == 'z');
EXPECT_TRUE(str_utf8_tolower(192) == 224); // À -> à
EXPECT_TRUE(str_utf8_tolower(7882) == 7883); // Ị -> ị
EXPECT_TRUE(str_utf8_comp_nocase("ÖlÜ", "ölü") == 0);
EXPECT_TRUE(str_utf8_comp_nocase("ÜlÖ", "ölü") > 0); // ü > ö
EXPECT_TRUE(str_utf8_comp_nocase("ÖlÜ", "ölüa") < 0); // NULL < a
EXPECT_TRUE(str_utf8_comp_nocase("ölüa", "ÖlÜ") > 0); // a < NULL
Run clang-format Purely automatic change. In case of conflict with this change, apply the other change and rerun the formatting to restore it: $ python scripts/fix_style.py
2020-09-26 19:41:58 +00:00
#if(CHAR_MIN < 0)
Fix negative numbers in char On some architectures char can be unsigned, which makes g++ very unhappy. error: narrowing conversion of '-128' from 'int' to 'char' inside { }
2019-04-02 15:19:20 +00:00
const char a[2] = {CHAR_MIN, 0};
UTF8 nocase compare & use for chat TAB completion - As suggested by Arseniy Zarche - Also updated confusables to Unicode 12
2019-01-07 22:49:20 +00:00
const char b[2] = {0, 0};
EXPECT_TRUE(str_utf8_comp_nocase(a, b) > 0);
EXPECT_TRUE(str_utf8_comp_nocase(b, a) < 0);
Fix negative numbers in char On some architectures char can be unsigned, which makes g++ very unhappy. error: narrowing conversion of '-128' from 'int' to 'char' inside { }
2019-04-02 15:19:20 +00:00
#endif
UTF8 nocase compare & use for chat TAB completion - As suggested by Arseniy Zarche - Also updated confusables to Unicode 12
2019-01-07 22:49:20 +00:00
Fix tests
2019-02-10 18:13:55 +00:00
EXPECT_TRUE(str_utf8_comp_nocase_num("ÖlÜ", "ölüa", 5) == 0);
EXPECT_TRUE(str_utf8_comp_nocase_num("ÖlÜ", "ölüa", 6) != 0);
UTF8 nocase compare & use for chat TAB completion - As suggested by Arseniy Zarche - Also updated confusables to Unicode 12
2019-01-07 22:49:20 +00:00
EXPECT_TRUE(str_utf8_comp_nocase_num("a", "z", 0) == 0);
EXPECT_TRUE(str_utf8_comp_nocase_num("a", "z", 1) != 0);
Add `end` output parameter to `str_utf8_find_nocase`, add tests Add optional `end` output parameter to `str_utf8_find_nocase` which is set to a pointer into the haystack after the matched string. This is necessary because there are pairs of matching upper case and lower case Unicode characters with different byte length (e.g. both `I` and `İ` map to `i`), so the byte length of the string matched in the haystack may not be identical to the length of the needle string.
2023-07-14 16:04:57 +00:00
}
TEST(Str, Utf8FindNocase)
{
const char *pStr = "abc";
const char *pEnd;
EXPECT_EQ(str_utf8_find_nocase(pStr, "a", &pEnd), pStr);
EXPECT_EQ(pEnd, pStr + str_length("a"));
EXPECT_EQ(str_utf8_find_nocase(pStr, "b", &pEnd), pStr + str_length("a"));
EXPECT_EQ(pEnd, pStr + str_length("ab"));
EXPECT_EQ(str_utf8_find_nocase(pStr, "c", &pEnd), pStr + str_length("ab"));
EXPECT_EQ(pEnd, pStr + str_length("abc"));
EXPECT_EQ(str_utf8_find_nocase(pStr, "d", &pEnd), nullptr);
EXPECT_EQ(pEnd, nullptr);
EXPECT_EQ(str_utf8_find_nocase(pStr, "A", &pEnd), pStr);
EXPECT_EQ(pEnd, pStr + str_length("a"));
EXPECT_EQ(str_utf8_find_nocase(pStr, "B", &pEnd), pStr + str_length("a"));
EXPECT_EQ(pEnd, pStr + str_length("ab"));
EXPECT_EQ(str_utf8_find_nocase(pStr, "C", &pEnd), pStr + str_length("ab"));
EXPECT_EQ(pEnd, pStr + str_length("abc"));
EXPECT_EQ(str_utf8_find_nocase(pStr, "D", &pEnd), nullptr);
EXPECT_EQ(pEnd, nullptr);
pStr = "ÄÖÜ";
EXPECT_EQ(str_utf8_find_nocase(pStr, "ä", &pEnd), pStr);
EXPECT_EQ(pEnd, pStr + str_length("Ä"));
EXPECT_EQ(str_utf8_find_nocase(pStr, "ö", &pEnd), pStr + str_length("Ä"));
EXPECT_EQ(pEnd, pStr + str_length("ÄÖ"));
EXPECT_EQ(str_utf8_find_nocase(pStr, "ü", &pEnd), pStr + str_length("ÄÖ"));
EXPECT_EQ(pEnd, pStr + str_length("ÄÖÜ"));
EXPECT_EQ(str_utf8_find_nocase(pStr, "z", &pEnd), nullptr);
EXPECT_EQ(pEnd, nullptr);
UTF8 nocase compare & use for chat TAB completion - As suggested by Arseniy Zarche - Also updated confusables to Unicode 12
2019-01-07 22:49:20 +00:00
Add `end` output parameter to `str_utf8_find_nocase`, add tests Add optional `end` output parameter to `str_utf8_find_nocase` which is set to a pointer into the haystack after the matched string. This is necessary because there are pairs of matching upper case and lower case Unicode characters with different byte length (e.g. both `I` and `İ` map to `i`), so the byte length of the string matched in the haystack may not be identical to the length of the needle string.
2023-07-14 16:04:57 +00:00
// Both 'I' and 'İ' map to 'i'
pStr = "antimatter";
EXPECT_EQ(str_utf8_find_nocase(pStr, "I", &pEnd), pStr + str_length("ant"));
EXPECT_EQ(pEnd, pStr + str_length("anti"));
EXPECT_EQ(str_utf8_find_nocase(pStr, "İ", &pEnd), pStr + str_length("ant"));
EXPECT_EQ(pEnd, pStr + str_length("anti"));
pStr = "ANTIMATTER";
EXPECT_EQ(str_utf8_find_nocase(pStr, "i", &pEnd), pStr + str_length("ANT"));
EXPECT_EQ(pEnd, pStr + str_length("ANTI"));
pStr = "ANTİMATTER";
EXPECT_EQ(str_utf8_find_nocase(pStr, "i", &pEnd), pStr + str_length("ANT"));
EXPECT_EQ(pEnd, pStr + str_length("ANTİ"));
UTF8 nocase compare & use for chat TAB completion - As suggested by Arseniy Zarche - Also updated confusables to Unicode 12
2019-01-07 22:49:20 +00:00
}
Add a function to fix UTF-8 truncation at the end of a string This can be used after truncating at byte boundaries.
2021-12-20 01:05:07 +00:00
TEST(Str, Utf8FixTruncation)
{
char aaBuf[][32] = {
"",
"\xff",
"abc",
"abc\xff",
"blub\xffxyz",
"привет Наташа\xff",
"до свидания\xffОлег",
};
const char *apExpected[] = {
"",
"",
"abc",
"abc",
"blub\xffxyz",
"привет Наташа",
"до свидания\xffОлег",
};
Use std::size instead of sizeof(a) / sizeof(a[0])
2022-03-30 13:16:19 +00:00
for(unsigned i = 0; i < std::size(aaBuf); i++)
Add a function to fix UTF-8 truncation at the end of a string This can be used after truncating at byte boundaries.
2021-12-20 01:05:07 +00:00
{
EXPECT_EQ(str_utf8_fix_truncation(aaBuf[i]), str_length(apExpected[i]));
EXPECT_STREQ(aaBuf[i], apExpected[i]);
}
}
Add `str_startswith` and `str_endswith` Also add a couple of tests for both.
2018-07-25 08:23:07 +00:00
TEST(Str, Startswith)
{
EXPECT_TRUE(str_startswith("abcdef", "abc"));
EXPECT_FALSE(str_startswith("abc", "abcdef"));
EXPECT_TRUE(str_startswith("xyz", ""));
EXPECT_FALSE(str_startswith("", "xyz"));
EXPECT_FALSE(str_startswith("house", "home"));
EXPECT_FALSE(str_startswith("blackboard", "board"));
EXPECT_TRUE(str_startswith("поплавать", "по"));
EXPECT_FALSE(str_startswith("плавать", "по"));
Return a pointer from `str_endswith` as well
2018-07-26 12:04:44 +00:00
static const char ABCDEFG[] = "abcdefg";
Add `str_startswith` and `str_endswith` Also add a couple of tests for both.
2018-07-25 08:23:07 +00:00
static const char ABC[] = "abc";
Return a pointer from `str_endswith` as well
2018-07-26 12:04:44 +00:00
EXPECT_EQ(str_startswith(ABCDEFG, ABC) - ABCDEFG, str_length(ABC));
Add `str_startswith` and `str_endswith` Also add a couple of tests for both.
2018-07-25 08:23:07 +00:00
}
Add str_startswith_nocase and str_endswith_nocase
2022-03-13 10:42:45 +00:00
TEST(Str, StartswithNocase)
{
EXPECT_TRUE(str_startswith_nocase("Abcdef", "abc"));
EXPECT_FALSE(str_startswith_nocase("aBc", "abcdef"));
EXPECT_TRUE(str_startswith_nocase("xYz", ""));
EXPECT_FALSE(str_startswith_nocase("", "xYz"));
EXPECT_FALSE(str_startswith_nocase("house", "home"));
EXPECT_FALSE(str_startswith_nocase("Blackboard", "board"));
EXPECT_TRUE(str_startswith_nocase("поплавать", "по"));
EXPECT_FALSE(str_startswith_nocase("плавать", "по"));
static const char ABCDEFG[] = "aBcdefg";
static const char ABC[] = "abc";
EXPECT_EQ(str_startswith_nocase(ABCDEFG, ABC) - ABCDEFG, str_length(ABC));
}
Add `str_startswith` and `str_endswith` Also add a couple of tests for both.
2018-07-25 08:23:07 +00:00
TEST(Str, Endswith)
{
EXPECT_TRUE(str_endswith("abcdef", "def"));
EXPECT_FALSE(str_endswith("def", "abcdef"));
EXPECT_TRUE(str_endswith("xyz", ""));
EXPECT_FALSE(str_endswith("", "xyz"));
EXPECT_FALSE(str_endswith("rhyme", "mine"));
EXPECT_FALSE(str_endswith("blackboard", "black"));
EXPECT_TRUE(str_endswith("люди", "юди"));
EXPECT_FALSE(str_endswith("люди", "любовь"));
Return a pointer from `str_endswith` as well
2018-07-26 12:04:44 +00:00
static const char ABCDEFG[] = "abcdefg";
static const char DEFG[] = "defg";
EXPECT_EQ(str_endswith(ABCDEFG, DEFG) - ABCDEFG,
str_length(ABCDEFG) - str_length(DEFG));
Add `str_startswith` and `str_endswith` Also add a couple of tests for both.
2018-07-25 08:23:07 +00:00
}
Also use sha256 for the mapbugs stuff
2018-08-19 10:12:11 +00:00
Allow translations to reorder string substitutions This is supported on Windows (https://docs.microsoft.com/en-us/cpp/c-runtime-library/printf-p-positional-parameters) and on POSIX, so basically everywhere. Add some tests to verify that the target system does indeed support these positional parameters. (cherry picked from commit ddd2b9319076efa81ca87e2f9582e328824f37db)
2019-02-10 16:13:13 +00:00
TEST(StrFormat, Positional)
{
char aBuf[256];
// normal
str_format(aBuf, sizeof(aBuf), "%s %s", "first", "second");
EXPECT_STREQ(aBuf, "first second");
// normal with positional arguments
str_format(aBuf, sizeof(aBuf), "%1$s %2$s", "first", "second");
EXPECT_STREQ(aBuf, "first second");
// reverse
str_format(aBuf, sizeof(aBuf), "%2$s %1$s", "first", "second");
EXPECT_STREQ(aBuf, "second first");
// duplicate
str_format(aBuf, sizeof(aBuf), "%1$s %1$s %2$d %1$s %2$d", "str", 1);
EXPECT_STREQ(aBuf, "str str 1 str 1");
}
Add str_startswith_nocase and str_endswith_nocase
2022-03-13 10:42:45 +00:00
TEST(Str, EndswithNocase)
{
EXPECT_TRUE(str_endswith_nocase("abcdef", "deF"));
EXPECT_FALSE(str_endswith_nocase("def", "abcdef"));
EXPECT_TRUE(str_endswith_nocase("xyz", ""));
EXPECT_FALSE(str_endswith_nocase("", "xyz"));
EXPECT_FALSE(str_endswith_nocase("rhyme", "minE"));
EXPECT_FALSE(str_endswith_nocase("blackboard", "black"));
EXPECT_TRUE(str_endswith_nocase("люди", "юди"));
EXPECT_FALSE(str_endswith_nocase("люди", "любовь"));
static const char ABCDEFG[] = "abcdefG";
static const char DEFG[] = "defg";
EXPECT_EQ(str_endswith_nocase(ABCDEFG, DEFG) - ABCDEFG,
str_length(ABCDEFG) - str_length(DEFG));
}
Fix `str_hex` and add tests - Properly null-terminate the destination buffer when the data size is zero. - Fix early loop exit when destination buffer is too small for all data.
2022-09-29 14:18:35 +00:00
TEST(Str, HexEncode)
{
char aOut[64];
const char *pData = "ABCD";
str_hex(aOut, sizeof(aOut), pData, 0);
EXPECT_STREQ(aOut, "");
str_hex(aOut, sizeof(aOut), pData, 1);
EXPECT_STREQ(aOut, "41 ");
str_hex(aOut, sizeof(aOut), pData, 2);
EXPECT_STREQ(aOut, "41 42 ");
str_hex(aOut, sizeof(aOut), pData, 3);
EXPECT_STREQ(aOut, "41 42 43 ");
str_hex(aOut, sizeof(aOut), pData, 4);
EXPECT_STREQ(aOut, "41 42 43 44 ");
Add `str_hex_cstyle` to print bytes as a C style array This function converts data to a C style array string presentation.
2023-01-17 23:09:07 +00:00
Fix `str_hex` and add tests - Properly null-terminate the destination buffer when the data size is zero. - Fix early loop exit when destination buffer is too small for all data.
2022-09-29 14:18:35 +00:00
str_hex(aOut, 1, pData, 4);
EXPECT_STREQ(aOut, "");
str_hex(aOut, 2, pData, 4);
EXPECT_STREQ(aOut, "");
str_hex(aOut, 3, pData, 4);
EXPECT_STREQ(aOut, "");
str_hex(aOut, 4, pData, 4);
EXPECT_STREQ(aOut, "41 ");
str_hex(aOut, 5, pData, 4);
EXPECT_STREQ(aOut, "41 ");
str_hex(aOut, 6, pData, 4);
EXPECT_STREQ(aOut, "41 ");
str_hex(aOut, 7, pData, 4);
EXPECT_STREQ(aOut, "41 42 ");
str_hex(aOut, 8, pData, 4);
EXPECT_STREQ(aOut, "41 42 ");
}
Add `str_hex_cstyle` to print bytes as a C style array This function converts data to a C style array string presentation.
2023-01-17 23:09:07 +00:00
TEST(Str, HexEncodeCstyle)
{
char aOut[128];
const char *pData = "ABCD";
str_hex_cstyle(aOut, sizeof(aOut), pData, 0);
EXPECT_STREQ(aOut, "");
str_hex_cstyle(aOut, sizeof(aOut), pData, 1);
EXPECT_STREQ(aOut, "0x41");
str_hex_cstyle(aOut, sizeof(aOut), pData, 2);
EXPECT_STREQ(aOut, "0x41, 0x42");
str_hex_cstyle(aOut, sizeof(aOut), pData, 3);
EXPECT_STREQ(aOut, "0x41, 0x42, 0x43");
str_hex_cstyle(aOut, sizeof(aOut), pData, 4);
EXPECT_STREQ(aOut, "0x41, 0x42, 0x43, 0x44");
str_hex_cstyle(aOut, 1, pData, 4);
EXPECT_STREQ(aOut, "");
str_hex_cstyle(aOut, 2, pData, 4);
EXPECT_STREQ(aOut, "");
str_hex_cstyle(aOut, 3, pData, 4);
EXPECT_STREQ(aOut, "");
str_hex_cstyle(aOut, 4, pData, 4);
EXPECT_STREQ(aOut, "");
str_hex_cstyle(aOut, 5, pData, 4);
EXPECT_STREQ(aOut, "");
str_hex_cstyle(aOut, 6, pData, 4);
EXPECT_STREQ(aOut, "");
str_hex_cstyle(aOut, 7, pData, 4);
EXPECT_STREQ(aOut, "0x41");
str_hex_cstyle(aOut, 12, pData, 4);
EXPECT_STREQ(aOut, "0x41");
str_hex_cstyle(aOut, 13, pData, 4);
EXPECT_STREQ(aOut, "0x41, 0x42");
str_hex_cstyle(aOut, 14, pData, 4);
EXPECT_STREQ(aOut, "0x41, 0x42");
str_hex_cstyle(aOut, sizeof(aOut), pData, 4, 1);
EXPECT_STREQ(aOut, "0x41,\n0x42,\n0x43,\n0x44");
str_hex_cstyle(aOut, sizeof(aOut), pData, 4, 2);
EXPECT_STREQ(aOut, "0x41, 0x42,\n0x43, 0x44");
str_hex_cstyle(aOut, sizeof(aOut), pData, 4, 3);
EXPECT_STREQ(aOut, "0x41, 0x42, 0x43,\n0x44");
str_hex_cstyle(aOut, sizeof(aOut), pData, 4, 4);
EXPECT_STREQ(aOut, "0x41, 0x42, 0x43, 0x44");
str_hex_cstyle(aOut, sizeof(aOut), pData, 4, 500);
EXPECT_STREQ(aOut, "0x41, 0x42, 0x43, 0x44");
}
Also use sha256 for the mapbugs stuff
2018-08-19 10:12:11 +00:00
TEST(Str, HexDecode)
{
char aOut[5] = {'a', 'b', 'c', 'd', 0};
Run clang-format Purely automatic change. In case of conflict with this change, apply the other change and rerun the formatting to restore it: $ python scripts/fix_style.py
2020-09-26 19:41:58 +00:00
EXPECT_EQ(str_hex_decode(aOut, 0, ""), 0);
EXPECT_STREQ(aOut, "abcd");
EXPECT_EQ(str_hex_decode(aOut, 0, " "), 2);
EXPECT_STREQ(aOut, "abcd");
EXPECT_EQ(str_hex_decode(aOut, 1, "1"), 2);
EXPECT_STREQ(aOut + 1, "bcd");
EXPECT_EQ(str_hex_decode(aOut, 1, "41"), 0);
EXPECT_STREQ(aOut, "Abcd");
EXPECT_EQ(str_hex_decode(aOut, 1, "4x"), 1);
EXPECT_STREQ(aOut + 1, "bcd");
EXPECT_EQ(str_hex_decode(aOut, 1, "x1"), 1);
EXPECT_STREQ(aOut + 1, "bcd");
EXPECT_EQ(str_hex_decode(aOut, 1, "411"), 2);
EXPECT_STREQ(aOut + 1, "bcd");
EXPECT_EQ(str_hex_decode(aOut, 4, "41424344"), 0);
EXPECT_STREQ(aOut, "ABCD");
Also use sha256 for the mapbugs stuff
2018-08-19 10:12:11 +00:00
}
Add str_tokenize and str_in_list
2019-02-06 12:30:47 +00:00
Add functions to encode/decode base64
2021-07-16 00:55:27 +00:00
void StrBase64Str(char *pBuffer, int BufferSize, const char *pString)
{
str_base64(pBuffer, BufferSize, pString, str_length(pString));
}
TEST(Str, Base64)
{
char aBuf[128];
str_base64(aBuf, sizeof(aBuf), "\0", 1);
EXPECT_STREQ(aBuf, "AA==");
str_base64(aBuf, sizeof(aBuf), "\0\0", 2);
EXPECT_STREQ(aBuf, "AAA=");
str_base64(aBuf, sizeof(aBuf), "\0\0\0", 3);
EXPECT_STREQ(aBuf, "AAAA");
StrBase64Str(aBuf, sizeof(aBuf), "");
EXPECT_STREQ(aBuf, "");
// https://en.wikipedia.org/w/index.php?title=Base64&oldid=1033503483#Output_padding
StrBase64Str(aBuf, sizeof(aBuf), "pleasure.");
EXPECT_STREQ(aBuf, "cGxlYXN1cmUu");
StrBase64Str(aBuf, sizeof(aBuf), "leasure.");
EXPECT_STREQ(aBuf, "bGVhc3VyZS4=");
StrBase64Str(aBuf, sizeof(aBuf), "easure.");
EXPECT_STREQ(aBuf, "ZWFzdXJlLg==");
StrBase64Str(aBuf, sizeof(aBuf), "asure.");
EXPECT_STREQ(aBuf, "YXN1cmUu");
StrBase64Str(aBuf, sizeof(aBuf), "sure.");
EXPECT_STREQ(aBuf, "c3VyZS4=");
}
TEST(Str, Base64Decode)
{
char aOut[17];
str_copy(aOut, "XXXXXXXXXXXXXXXX", sizeof(aOut));
EXPECT_EQ(str_base64_decode(aOut, sizeof(aOut), ""), 0);
EXPECT_STREQ(aOut, "XXXXXXXXXXXXXXXX");
// https://en.wikipedia.org/w/index.php?title=Base64&oldid=1033503483#Output_padding
str_copy(aOut, "XXXXXXXXXXXXXXXX", sizeof(aOut));
EXPECT_EQ(str_base64_decode(aOut, sizeof(aOut), "cGxlYXN1cmUu"), 9);
EXPECT_STREQ(aOut, "pleasure.XXXXXXX");
str_copy(aOut, "XXXXXXXXXXXXXXXX", sizeof(aOut));
EXPECT_EQ(str_base64_decode(aOut, sizeof(aOut), "bGVhc3VyZS4="), 8);
EXPECT_STREQ(aOut, "leasure.XXXXXXXX");
str_copy(aOut, "XXXXXXXXXXXXXXXX", sizeof(aOut));
EXPECT_EQ(str_base64_decode(aOut, sizeof(aOut), "ZWFzdXJlLg=="), 7);
EXPECT_STREQ(aOut, "easure.XXXXXXXXX");
str_copy(aOut, "XXXXXXXXXXXXXXXX", sizeof(aOut));
EXPECT_EQ(str_base64_decode(aOut, sizeof(aOut), "YXN1cmUu"), 6);
EXPECT_STREQ(aOut, "asure.XXXXXXXXXX");
str_copy(aOut, "XXXXXXXXXXXXXXXX", sizeof(aOut));
EXPECT_EQ(str_base64_decode(aOut, sizeof(aOut), "c3VyZS4="), 5);
EXPECT_STREQ(aOut, "sure.XXXXXXXXXXX");
str_copy(aOut, "XXXXXXXXXXXXXXXX", sizeof(aOut));
EXPECT_EQ(str_base64_decode(aOut, sizeof(aOut), "////"), 3);
EXPECT_STREQ(aOut, "\xff\xff\xffXXXXXXXXXXXXX");
}
TEST(Str, Base64DecodeError)
{
char aBuf[128];
// Wrong padding.
EXPECT_LT(str_base64_decode(aBuf, sizeof(aBuf), "A"), 0);
EXPECT_LT(str_base64_decode(aBuf, sizeof(aBuf), "AA"), 0);
EXPECT_LT(str_base64_decode(aBuf, sizeof(aBuf), "AAA"), 0);
EXPECT_LT(str_base64_decode(aBuf, sizeof(aBuf), "A==="), 0);
EXPECT_LT(str_base64_decode(aBuf, sizeof(aBuf), "=AAA"), 0);
EXPECT_LT(str_base64_decode(aBuf, sizeof(aBuf), "===="), 0);
EXPECT_LT(str_base64_decode(aBuf, sizeof(aBuf), "AAA=AAAA"), 0);
// Invalid characters.
EXPECT_LT(str_base64_decode(aBuf, sizeof(aBuf), "----"), 0);
EXPECT_LT(str_base64_decode(aBuf, sizeof(aBuf), "AAAA "), 0);
EXPECT_LT(str_base64_decode(aBuf, sizeof(aBuf), "AAA "), 0);
// Invalid padding values.
EXPECT_LT(str_base64_decode(aBuf, sizeof(aBuf), "//=="), 0);
}
Don't use strtok. Close #1468
2019-03-05 09:46:29 +00:00
TEST(Str, Tokenize)
{
char aTest[] = "GER,RUS,ZAF,BRA,CAN";
Huge variable naming format Fix pointer and pointer array variable naming Huge renaming to match our rules Used regex: (?!(return|delete)\b)\b\w+ (m_|ms_|g_|gs_|s_)[^a]\w+\[ (?!(return|delete)\b)\b\w+ (?!(m_|ms_|g_|gs_|s_))[^a]\w+\[ Further format static variables Format almost all pointer names accordingly Used regex: (?!(return)\b)\b\w+ \*(?!(m_p|p|s_p|m_ap|s_ap|g_p|g_ap|ap|gs_ap|ms_ap|gs_p|ms_p))\w+\b[^:\(p] clang-format Fix CI fail Fix misnamed non pointer as pointer and non array as array Used regex: (?!(return|delete)\b)\b\w+ (m_|ms_|g_|gs_|s_)p\w+\b (?!return\b)\b\w+ (ms_|m_|g_|gs_|s_)a\w+\b[^\[] clang-format Revert to SCREAMING_SNAKE_CASE and reinstate dead code
2022-06-30 22:36:32 +00:00
const char *apOut[] = {"GER", "RUS", "ZAF", "BRA", "CAN"};
Don't use strtok. Close #1468
2019-03-05 09:46:29 +00:00
char aBuf[4];
int n = 0;
Huge variable naming format Fix pointer and pointer array variable naming Huge renaming to match our rules Used regex: (?!(return|delete)\b)\b\w+ (m_|ms_|g_|gs_|s_)[^a]\w+\[ (?!(return|delete)\b)\b\w+ (?!(m_|ms_|g_|gs_|s_))[^a]\w+\[ Further format static variables Format almost all pointer names accordingly Used regex: (?!(return)\b)\b\w+ \*(?!(m_p|p|s_p|m_ap|s_ap|g_p|g_ap|ap|gs_ap|ms_ap|gs_p|ms_p))\w+\b[^:\(p] clang-format Fix CI fail Fix misnamed non pointer as pointer and non array as array Used regex: (?!(return|delete)\b)\b\w+ (m_|ms_|g_|gs_|s_)p\w+\b (?!return\b)\b\w+ (ms_|m_|g_|gs_|s_)a\w+\b[^\[] clang-format Revert to SCREAMING_SNAKE_CASE and reinstate dead code
2022-06-30 22:36:32 +00:00
for(const char *pTok = aTest; (pTok = str_next_token(pTok, ",", aBuf, sizeof(aBuf)));)
EXPECT_STREQ(apOut[n++], aBuf);
Don't use strtok. Close #1468
2019-03-05 09:46:29 +00:00
char aTest2[] = "";
Style
2019-03-11 11:54:31 +00:00
EXPECT_EQ(str_next_token(aTest2, ",", aBuf, sizeof(aBuf)), nullptr);
Work on compound binds
2019-04-28 13:32:14 +00:00
char aTest3[] = "+b";
str_next_token(aTest3, "+", aBuf, sizeof(aBuf));
EXPECT_STREQ(aBuf, "b");
Don't use strtok. Close #1468
2019-03-05 09:46:29 +00:00
}
Add str_tokenize and str_in_list
2019-02-06 12:30:47 +00:00
TEST(Str, InList)
{
char aTest[] = "GER,RUS,ZAF,BRA,CAN";
EXPECT_TRUE(str_in_list(aTest, ",", "GER"));
EXPECT_TRUE(str_in_list(aTest, ",", "RUS"));
EXPECT_TRUE(str_in_list(aTest, ",", "ZAF"));
EXPECT_TRUE(str_in_list(aTest, ",", "BRA"));
EXPECT_TRUE(str_in_list(aTest, ",", "CAN"));
EXPECT_FALSE(str_in_list(aTest, ",", "CHN"));
EXPECT_FALSE(str_in_list(aTest, ",", "R,R"));
Add a couple tests, fix #1457
2019-02-13 14:11:09 +00:00
EXPECT_FALSE(str_in_list("abc,xyz", ",", "abcdef"));
EXPECT_FALSE(str_in_list("", ",", ""));
EXPECT_FALSE(str_in_list("", ",", "xyz"));
Render news without STL functions (fixes #1693)
2019-04-29 20:55:12 +00:00
EXPECT_TRUE(str_in_list("FOO,,BAR", ",", ""));
Prevent NULL pointer in mem_copy (fixes #2753)
2020-09-06 14:29:20 +00:00
EXPECT_TRUE(str_in_list("abc,,def", ",", "def"));
Add str_tokenize and str_in_list
2019-02-06 12:30:47 +00:00
}
str_format: always return length of written string Quoting the man page of vsnprintf: RETURN VALUE Upon successful return, these functions return the number of characters printed (excluding the null byte used to end output to strings). The functions snprintf() and vsnprintf() do not write more than size bytes (including the terminating null byte ('\0')). If the output was truncated due to this limit, then the return value is the number of characters (excluding the terminating null byte) which would have been written to the final string if enough space had been available. Thus, a return value of size or more means that the output was truncated. (See also below under NOTES.) If an output error is encountered, a negative value is returned. [...] The glibc implementation of the functions snprintf() and vsnprintf() conforms to the C99 standard, that is, behaves as described above, since glibc version 2.1. Until glibc 2.0.6, they would return -1 when the output was truncated.
2019-03-20 18:09:23 +00:00
Unify string test naming
2022-09-11 21:40:32 +00:00
TEST(Str, Format)
str_format: always return length of written string Quoting the man page of vsnprintf: RETURN VALUE Upon successful return, these functions return the number of characters printed (excluding the null byte used to end output to strings). The functions snprintf() and vsnprintf() do not write more than size bytes (including the terminating null byte ('\0')). If the output was truncated due to this limit, then the return value is the number of characters (excluding the terminating null byte) which would have been written to the final string if enough space had been available. Thus, a return value of size or more means that the output was truncated. (See also below under NOTES.) If an output error is encountered, a negative value is returned. [...] The glibc implementation of the functions snprintf() and vsnprintf() conforms to the C99 standard, that is, behaves as described above, since glibc version 2.1. Until glibc 2.0.6, they would return -1 when the output was truncated.
2019-03-20 18:09:23 +00:00
{
char aBuf[4];
EXPECT_EQ(str_format(aBuf, 4, "%d:", 9), 2);
EXPECT_STREQ(aBuf, "9:");
EXPECT_EQ(str_format(aBuf, 4, "%d: ", 9), 3);
EXPECT_STREQ(aBuf, "9: ");
EXPECT_EQ(str_format(aBuf, 4, "%d: ", 99), 3);
EXPECT_STREQ(aBuf, "99:");
}
Names can include quotes :/
2019-04-16 00:24:24 +00:00
Unify string test naming
2022-09-11 21:40:32 +00:00
TEST(Str, FormatTruncate)
Fix string functions not to truncate in the middle of a UTF-8 character This makes `str_utf8_copy` obsolete.
2021-12-20 01:34:02 +00:00
{
const char *pStr = "DDNet最好了";
char aBuf[64];
str_format(aBuf, 7, "%s", pStr);
EXPECT_STREQ(aBuf, "DDNet");
str_format(aBuf, 8, "%s", pStr);
EXPECT_STREQ(aBuf, "DDNet");
str_format(aBuf, 9, "%s", pStr);
EXPECT_STREQ(aBuf, "DDNet最");
str_format(aBuf, 10, "%s", pStr);
EXPECT_STREQ(aBuf, "DDNet最");
str_format(aBuf, 11, "%s", pStr);
EXPECT_STREQ(aBuf, "DDNet最");
str_format(aBuf, 12, "%s", pStr);
EXPECT_STREQ(aBuf, "DDNet最好");
str_format(aBuf, 13, "%s", pStr);
EXPECT_STREQ(aBuf, "DDNet最好");
str_format(aBuf, 14, "%s", pStr);
EXPECT_STREQ(aBuf, "DDNet最好");
str_format(aBuf, 15, "%s", pStr);
EXPECT_STREQ(aBuf, "DDNet最好了");
str_format(aBuf, 16, "%s", pStr);
EXPECT_STREQ(aBuf, "DDNet最好了");
}
Improve `str_trim_words` and add tests - Change argument and return value to `const char *`, as the string is not modified by this function. - Use our own `str_isspace` instead of standard library `isspace`. - Always trim leading whitespace to correctly handle inputs with leading whitespace.
2022-09-28 11:53:05 +00:00
TEST(Str, TrimWords)
{
const char *pStr1 = "aa bb ccc dddd eeeee";
EXPECT_STREQ(str_trim_words(pStr1, 0), "aa bb ccc dddd eeeee");
EXPECT_STREQ(str_trim_words(pStr1, 1), "bb ccc dddd eeeee");
EXPECT_STREQ(str_trim_words(pStr1, 2), "ccc dddd eeeee");
EXPECT_STREQ(str_trim_words(pStr1, 3), "dddd eeeee");
EXPECT_STREQ(str_trim_words(pStr1, 4), "eeeee");
EXPECT_STREQ(str_trim_words(pStr1, 5), "");
EXPECT_STREQ(str_trim_words(pStr1, 100), "");
const char *pStr2 = " aaa bb ";
EXPECT_STREQ(str_trim_words(pStr2, 0), "aaa bb ");
EXPECT_STREQ(str_trim_words(pStr2, 1), "bb ");
EXPECT_STREQ(str_trim_words(pStr2, 2), "");
EXPECT_STREQ(str_trim_words(pStr2, 100), "");
const char *pStr3 = "\n\naa bb\t\tccc\r\n\r\ndddd";
EXPECT_STREQ(str_trim_words(pStr3, 0), "aa bb\t\tccc\r\n\r\ndddd");
EXPECT_STREQ(str_trim_words(pStr3, 1), "bb\t\tccc\r\n\r\ndddd");
EXPECT_STREQ(str_trim_words(pStr3, 2), "ccc\r\n\r\ndddd");
EXPECT_STREQ(str_trim_words(pStr3, 3), "dddd");
EXPECT_STREQ(str_trim_words(pStr3, 4), "");
EXPECT_STREQ(str_trim_words(pStr3, 100), "");
}
Unify string test naming
2022-09-11 21:40:32 +00:00
TEST(Str, CopyNum)
Names can include quotes :/
2019-04-16 00:24:24 +00:00
{
Huge variable naming format Fix pointer and pointer array variable naming Huge renaming to match our rules Used regex: (?!(return|delete)\b)\b\w+ (m_|ms_|g_|gs_|s_)[^a]\w+\[ (?!(return|delete)\b)\b\w+ (?!(m_|ms_|g_|gs_|s_))[^a]\w+\[ Further format static variables Format almost all pointer names accordingly Used regex: (?!(return)\b)\b\w+ \*(?!(m_p|p|s_p|m_ap|s_ap|g_p|g_ap|ap|gs_ap|ms_ap|gs_p|ms_p))\w+\b[^:\(p] clang-format Fix CI fail Fix misnamed non pointer as pointer and non array as array Used regex: (?!(return|delete)\b)\b\w+ (m_|ms_|g_|gs_|s_)p\w+\b (?!return\b)\b\w+ (ms_|m_|g_|gs_|s_)a\w+\b[^\[] clang-format Revert to SCREAMING_SNAKE_CASE and reinstate dead code
2022-06-30 22:36:32 +00:00
const char *pFoo = "Foobaré";
Names can include quotes :/
2019-04-16 00:24:24 +00:00
char aBuf[64];
Huge variable naming format Fix pointer and pointer array variable naming Huge renaming to match our rules Used regex: (?!(return|delete)\b)\b\w+ (m_|ms_|g_|gs_|s_)[^a]\w+\[ (?!(return|delete)\b)\b\w+ (?!(m_|ms_|g_|gs_|s_))[^a]\w+\[ Further format static variables Format almost all pointer names accordingly Used regex: (?!(return)\b)\b\w+ \*(?!(m_p|p|s_p|m_ap|s_ap|g_p|g_ap|ap|gs_ap|ms_ap|gs_p|ms_p))\w+\b[^:\(p] clang-format Fix CI fail Fix misnamed non pointer as pointer and non array as array Used regex: (?!(return|delete)\b)\b\w+ (m_|ms_|g_|gs_|s_)p\w+\b (?!return\b)\b\w+ (ms_|m_|g_|gs_|s_)a\w+\b[^\[] clang-format Revert to SCREAMING_SNAKE_CASE and reinstate dead code
2022-06-30 22:36:32 +00:00
str_utf8_truncate(aBuf, 3, pFoo, 1);
Names can include quotes :/
2019-04-16 00:24:24 +00:00
EXPECT_STREQ(aBuf, "F");
Huge variable naming format Fix pointer and pointer array variable naming Huge renaming to match our rules Used regex: (?!(return|delete)\b)\b\w+ (m_|ms_|g_|gs_|s_)[^a]\w+\[ (?!(return|delete)\b)\b\w+ (?!(m_|ms_|g_|gs_|s_))[^a]\w+\[ Further format static variables Format almost all pointer names accordingly Used regex: (?!(return)\b)\b\w+ \*(?!(m_p|p|s_p|m_ap|s_ap|g_p|g_ap|ap|gs_ap|ms_ap|gs_p|ms_p))\w+\b[^:\(p] clang-format Fix CI fail Fix misnamed non pointer as pointer and non array as array Used regex: (?!(return|delete)\b)\b\w+ (m_|ms_|g_|gs_|s_)p\w+\b (?!return\b)\b\w+ (ms_|m_|g_|gs_|s_)a\w+\b[^\[] clang-format Revert to SCREAMING_SNAKE_CASE and reinstate dead code
2022-06-30 22:36:32 +00:00
str_utf8_truncate(aBuf, 3, pFoo, 2);
Names can include quotes :/
2019-04-16 00:24:24 +00:00
EXPECT_STREQ(aBuf, "Fo");
Huge variable naming format Fix pointer and pointer array variable naming Huge renaming to match our rules Used regex: (?!(return|delete)\b)\b\w+ (m_|ms_|g_|gs_|s_)[^a]\w+\[ (?!(return|delete)\b)\b\w+ (?!(m_|ms_|g_|gs_|s_))[^a]\w+\[ Further format static variables Format almost all pointer names accordingly Used regex: (?!(return)\b)\b\w+ \*(?!(m_p|p|s_p|m_ap|s_ap|g_p|g_ap|ap|gs_ap|ms_ap|gs_p|ms_p))\w+\b[^:\(p] clang-format Fix CI fail Fix misnamed non pointer as pointer and non array as array Used regex: (?!(return|delete)\b)\b\w+ (m_|ms_|g_|gs_|s_)p\w+\b (?!return\b)\b\w+ (ms_|m_|g_|gs_|s_)a\w+\b[^\[] clang-format Revert to SCREAMING_SNAKE_CASE and reinstate dead code
2022-06-30 22:36:32 +00:00
str_utf8_truncate(aBuf, 3, pFoo, 3);
Names can include quotes :/
2019-04-16 00:24:24 +00:00
EXPECT_STREQ(aBuf, "Fo");
Huge variable naming format Fix pointer and pointer array variable naming Huge renaming to match our rules Used regex: (?!(return|delete)\b)\b\w+ (m_|ms_|g_|gs_|s_)[^a]\w+\[ (?!(return|delete)\b)\b\w+ (?!(m_|ms_|g_|gs_|s_))[^a]\w+\[ Further format static variables Format almost all pointer names accordingly Used regex: (?!(return)\b)\b\w+ \*(?!(m_p|p|s_p|m_ap|s_ap|g_p|g_ap|ap|gs_ap|ms_ap|gs_p|ms_p))\w+\b[^:\(p] clang-format Fix CI fail Fix misnamed non pointer as pointer and non array as array Used regex: (?!(return|delete)\b)\b\w+ (m_|ms_|g_|gs_|s_)p\w+\b (?!return\b)\b\w+ (ms_|m_|g_|gs_|s_)a\w+\b[^\[] clang-format Revert to SCREAMING_SNAKE_CASE and reinstate dead code
2022-06-30 22:36:32 +00:00
str_utf8_truncate(aBuf, sizeof(aBuf), pFoo, 6);
Names can include quotes :/
2019-04-16 00:24:24 +00:00
EXPECT_STREQ(aBuf, "Foobar");
Huge variable naming format Fix pointer and pointer array variable naming Huge renaming to match our rules Used regex: (?!(return|delete)\b)\b\w+ (m_|ms_|g_|gs_|s_)[^a]\w+\[ (?!(return|delete)\b)\b\w+ (?!(m_|ms_|g_|gs_|s_))[^a]\w+\[ Further format static variables Format almost all pointer names accordingly Used regex: (?!(return)\b)\b\w+ \*(?!(m_p|p|s_p|m_ap|s_ap|g_p|g_ap|ap|gs_ap|ms_ap|gs_p|ms_p))\w+\b[^:\(p] clang-format Fix CI fail Fix misnamed non pointer as pointer and non array as array Used regex: (?!(return|delete)\b)\b\w+ (m_|ms_|g_|gs_|s_)p\w+\b (?!return\b)\b\w+ (ms_|m_|g_|gs_|s_)a\w+\b[^\[] clang-format Revert to SCREAMING_SNAKE_CASE and reinstate dead code
2022-06-30 22:36:32 +00:00
str_utf8_truncate(aBuf, sizeof(aBuf), pFoo, 7);
Fix utf8 handling in str_truncate (fixes #2533) and rename to str_utf8_truncate since it's only used for utf8 strings
2020-08-02 11:45:20 +00:00
EXPECT_STREQ(aBuf, "Foobaré");
Huge variable naming format Fix pointer and pointer array variable naming Huge renaming to match our rules Used regex: (?!(return|delete)\b)\b\w+ (m_|ms_|g_|gs_|s_)[^a]\w+\[ (?!(return|delete)\b)\b\w+ (?!(m_|ms_|g_|gs_|s_))[^a]\w+\[ Further format static variables Format almost all pointer names accordingly Used regex: (?!(return)\b)\b\w+ \*(?!(m_p|p|s_p|m_ap|s_ap|g_p|g_ap|ap|gs_ap|ms_ap|gs_p|ms_p))\w+\b[^:\(p] clang-format Fix CI fail Fix misnamed non pointer as pointer and non array as array Used regex: (?!(return|delete)\b)\b\w+ (m_|ms_|g_|gs_|s_)p\w+\b (?!return\b)\b\w+ (ms_|m_|g_|gs_|s_)a\w+\b[^\[] clang-format Revert to SCREAMING_SNAKE_CASE and reinstate dead code
2022-06-30 22:36:32 +00:00
str_utf8_truncate(aBuf, sizeof(aBuf), pFoo, 0);
Add str_utf8_truncate test for size 0
2021-01-22 18:17:56 +00:00
EXPECT_STREQ(aBuf, "");
Fix utf8 handling in str_truncate (fixes #2533) and rename to str_utf8_truncate since it's only used for utf8 strings
2020-08-02 11:45:20 +00:00
char aBuf2[8];
Huge variable naming format Fix pointer and pointer array variable naming Huge renaming to match our rules Used regex: (?!(return|delete)\b)\b\w+ (m_|ms_|g_|gs_|s_)[^a]\w+\[ (?!(return|delete)\b)\b\w+ (?!(m_|ms_|g_|gs_|s_))[^a]\w+\[ Further format static variables Format almost all pointer names accordingly Used regex: (?!(return)\b)\b\w+ \*(?!(m_p|p|s_p|m_ap|s_ap|g_p|g_ap|ap|gs_ap|ms_ap|gs_p|ms_p))\w+\b[^:\(p] clang-format Fix CI fail Fix misnamed non pointer as pointer and non array as array Used regex: (?!(return|delete)\b)\b\w+ (m_|ms_|g_|gs_|s_)p\w+\b (?!return\b)\b\w+ (ms_|m_|g_|gs_|s_)a\w+\b[^\[] clang-format Revert to SCREAMING_SNAKE_CASE and reinstate dead code
2022-06-30 22:36:32 +00:00
str_utf8_truncate(aBuf2, sizeof(aBuf2), pFoo, 7);
Fix utf8 handling in str_truncate (fixes #2533) and rename to str_utf8_truncate since it's only used for utf8 strings
2020-08-02 11:45:20 +00:00
EXPECT_STREQ(aBuf2, "Foobar");
char aBuf3[9];
Huge variable naming format Fix pointer and pointer array variable naming Huge renaming to match our rules Used regex: (?!(return|delete)\b)\b\w+ (m_|ms_|g_|gs_|s_)[^a]\w+\[ (?!(return|delete)\b)\b\w+ (?!(m_|ms_|g_|gs_|s_))[^a]\w+\[ Further format static variables Format almost all pointer names accordingly Used regex: (?!(return)\b)\b\w+ \*(?!(m_p|p|s_p|m_ap|s_ap|g_p|g_ap|ap|gs_ap|ms_ap|gs_p|ms_p))\w+\b[^:\(p] clang-format Fix CI fail Fix misnamed non pointer as pointer and non array as array Used regex: (?!(return|delete)\b)\b\w+ (m_|ms_|g_|gs_|s_)p\w+\b (?!return\b)\b\w+ (ms_|m_|g_|gs_|s_)a\w+\b[^\[] clang-format Revert to SCREAMING_SNAKE_CASE and reinstate dead code
2022-06-30 22:36:32 +00:00
str_utf8_truncate(aBuf3, sizeof(aBuf3), pFoo, 7);
Fix utf8 handling in str_truncate (fixes #2533) and rename to str_utf8_truncate since it's only used for utf8 strings
2020-08-02 11:45:20 +00:00
EXPECT_STREQ(aBuf3, "Foobaré");
Names can include quotes :/
2019-04-16 00:24:24 +00:00
}
Add test cases
2020-09-06 22:57:21 +00:00
Unify string test naming
2022-09-11 21:40:32 +00:00
TEST(Str, Copy)
Add test cases
2020-09-06 22:57:21 +00:00
{
Fix string functions not to truncate in the middle of a UTF-8 character This makes `str_utf8_copy` obsolete.
2021-12-20 01:34:02 +00:00
const char *pStr = "DDNet最好了";
Add test cases
2020-09-06 22:57:21 +00:00
char aBuf[64];
Fix string functions not to truncate in the middle of a UTF-8 character This makes `str_utf8_copy` obsolete.
2021-12-20 01:34:02 +00:00
str_copy(aBuf, pStr, 7);
Add test cases
2020-09-06 22:57:21 +00:00
EXPECT_STREQ(aBuf, "DDNet");
Fix string functions not to truncate in the middle of a UTF-8 character This makes `str_utf8_copy` obsolete.
2021-12-20 01:34:02 +00:00
str_copy(aBuf, pStr, 8);
Add test cases
2020-09-06 22:57:21 +00:00
EXPECT_STREQ(aBuf, "DDNet");
Fix string functions not to truncate in the middle of a UTF-8 character This makes `str_utf8_copy` obsolete.
2021-12-20 01:34:02 +00:00
str_copy(aBuf, pStr, 9);
I can't count
2020-09-06 23:03:40 +00:00
EXPECT_STREQ(aBuf, "DDNet最");
Fix string functions not to truncate in the middle of a UTF-8 character This makes `str_utf8_copy` obsolete.
2021-12-20 01:34:02 +00:00
str_copy(aBuf, pStr, 10);
Add test cases
2020-09-06 22:57:21 +00:00
EXPECT_STREQ(aBuf, "DDNet最");
Fix string functions not to truncate in the middle of a UTF-8 character This makes `str_utf8_copy` obsolete.
2021-12-20 01:34:02 +00:00
str_copy(aBuf, pStr, 11);
Add test cases
2020-09-06 22:57:21 +00:00
EXPECT_STREQ(aBuf, "DDNet最");
Fix string functions not to truncate in the middle of a UTF-8 character This makes `str_utf8_copy` obsolete.
2021-12-20 01:34:02 +00:00
str_copy(aBuf, pStr, 12);
I can't count
2020-09-06 23:03:40 +00:00
EXPECT_STREQ(aBuf, "DDNet最好");
Fix string functions not to truncate in the middle of a UTF-8 character This makes `str_utf8_copy` obsolete.
2021-12-20 01:34:02 +00:00
str_copy(aBuf, pStr, 13);
Add test cases
2020-09-06 22:57:21 +00:00
EXPECT_STREQ(aBuf, "DDNet最好");
Fix string functions not to truncate in the middle of a UTF-8 character This makes `str_utf8_copy` obsolete.
2021-12-20 01:34:02 +00:00
str_copy(aBuf, pStr, 14);
Add test cases
2020-09-06 22:57:21 +00:00
EXPECT_STREQ(aBuf, "DDNet最好");
Fix string functions not to truncate in the middle of a UTF-8 character This makes `str_utf8_copy` obsolete.
2021-12-20 01:34:02 +00:00
str_copy(aBuf, pStr, 15);
I can't count
2020-09-06 23:03:40 +00:00
EXPECT_STREQ(aBuf, "DDNet最好了");
Fix string functions not to truncate in the middle of a UTF-8 character This makes `str_utf8_copy` obsolete.
2021-12-20 01:34:02 +00:00
str_copy(aBuf, pStr, 16);
Add test cases
2020-09-06 22:57:21 +00:00
EXPECT_STREQ(aBuf, "DDNet最好了");
Add tests for `str_append` and for templated `str_copy`
2023-06-13 22:03:30 +00:00
str_copy(aBuf, pStr);
EXPECT_STREQ(aBuf, "DDNet最好了");
}
TEST(Str, Append)
{
char aBuf[64];
aBuf[0] = '\0';
str_append(aBuf, "DDNet最好了", 7);
EXPECT_STREQ(aBuf, "DDNet");
str_append(aBuf, "最", 8);
EXPECT_STREQ(aBuf, "DDNet");
str_append(aBuf, "最", 9);
EXPECT_STREQ(aBuf, "DDNet最");
str_append(aBuf, "好", 10);
EXPECT_STREQ(aBuf, "DDNet最");
str_append(aBuf, "好", 11);
EXPECT_STREQ(aBuf, "DDNet最");
str_append(aBuf, "好", 12);
EXPECT_STREQ(aBuf, "DDNet最好");
str_append(aBuf, "了", 13);
EXPECT_STREQ(aBuf, "DDNet最好");
str_append(aBuf, "了", 14);
EXPECT_STREQ(aBuf, "DDNet最好");
str_append(aBuf, "了", 15);
EXPECT_STREQ(aBuf, "DDNet最好了");
str_append(aBuf, "了", 16);
EXPECT_STREQ(aBuf, "DDNet最好了");
aBuf[0] = '\0';
str_append(aBuf, "DDNet最好了");
EXPECT_STREQ(aBuf, "DDNet最好了");
Add test cases
2020-09-06 22:57:21 +00:00
}
Add str_time/str_time_float Handles centisecs < 0 Uses int64, otherwise we're limited to ~260 days
2020-10-18 20:44:02 +00:00
add str_utf8_stats to base system
2021-11-24 22:24:46 +00:00
TEST(Str, Utf8Stats)
{
Port line input and IME support from 0.7 Port the line input (UI edit boxes, chat, console) and Input Method Editor (IME) support from upstream. Closes #4397. General ------------------------------ Fix issues with the text input. Closes #4346. Closes #4524. Word skipping (when holding Ctrl) is overhauled to be consistent with the Windows / Firefox experience that I took as reference. Improve usability by not blinking (i.e. always rendering) the caret shortly after is has been moved. UI text input ------------------------------ Fix inconsistent mouse-based left and right scrolling (closes #4347). Support smooth left and right scrolling. Chat ------------------------------ Support keyboard-based text selection of the chat input. Mouse-based selection could be support in the future when we decide to add something like an ingame UI cursor. Support smooth up and down scrolling of the chat input, removing the old hack that offsets the input string to simulate scrolling. Console ------------------------------ Also support mouse-based text selection of the command input. Only text from either the command input or the console log can be selected at the same time. This ensures that Ctrl+C will always copy the text that is currently visually selected in the console. Check for Ctrl+C input event in event handler instead of in render function, to hopefully fix the issue that copying does not work sometimes (closes #5974 until further notice). When Ctrl+C is used to copy text from the console log, the selection is cleared. This should make it more clear when text was copied from the log. Fix an issue that was preventing the console log selection from being cleared, when all log lines are selected. Remove Ctrl+A/E hotkeys that move cursor to beginning/end respectively. Ctrl+A now selectes all text like for all other inputs. Home and End keys can still be used to go the beginning and end. Remove Ctrl+U/K hotkeys that clear everything before/after the cursor respectively. Hold shift and use Home/End to select everything instead. IME support ------------------------------ Render list of IME candidates in the client on Windows, so the candidate list can also be viewed in fullscreen mode. There is no API available to retrieve a candidate list on the other operating systems. Improve composition rendering by underlining the composition text instead of putting it in square brackets. Track active input globally to properly activate and deactivate IME through the SDL functions. Closes #1030. Closes #1008. Password rendering ------------------------------ Fix rendering of passwords containing unicode. Instead of rendering one star character for each UTF-8 `char`, render on star for every unicode codepoint. Show the composition text also for passwords. Without seeing the composition text it's hard to type a password containing those characters. The candidate window exposes the composition anyway. If you don't want to expose your password this way, e.g. while streaming, you could: 1. Use a latin password and switch off the IME for the password input with the IME hotkey. 2. Blank your screen with an external program while you are streaming and entering passwords. 3. Use binds to authenticate in rcon or to set the server browser password. Refactoring ------------------------------ Move all text input logic and general rendering to `CLineInput`. A `CLineInput` is associated with a particular `char` buffer given as a pointer either in the constructor or with `SetBuffer`. The maximum byte size of the buffer must also be specified. The maximum length in unicode codepoints can also be specified separately (e.g. on upstream, name are limited by the number of unicode codepoints instead). Add `CLineInputBuffered`, which is a `CLineInput` that own a `char` buffer of a fixed size, which is specified as a template argument. As `CLineInput` does not own a buffer anymore, this reduces duplicate code for line inputs that need their own buffer. Add `CLineInputNumber` which has additional convenience functions to consider the text as an `int` or `float`, to reduce duplicate code in those cases. In the future we could also add an input filter function so that only numbers can be entered in the number input. Add `CLineInput::SetClipboardLineCallback` to handle the case that multiple lines of text are pasted into a lineinput. This reduces duplicate code, as this behavior was previously implemented separately for chat and console. The behavior is also fixed to be consistent with the console on Windows, so the first line being pasted edits the current input text and then sends it instead of being sent on its own without the existing input text. Add `CalcFontSizeAndBoundingBox` to UI to reduce duplicate code. Expose `CalcAlignedCursorPos` as static member function to reuse it for line input. Dispatch input events to UI inputs through the event handler instead of storing them in a duplicate buffer. Use `size_t` for line input cursor position, length etc. and for `str_utf8_stats`. Add `IButtonColorFunction` to UI to describe a functions that defines colors for the Default, Active and Hovered states of UI elements. Add some default button color functions. Use button color function to reduce duplicate code in scrollbar rendering. Use `vec2` instead of two `floats` to represent the mouse positions in the text renderer. Remove `CaretPosition` again, as it does not calculate the correct Y position near line breaks due to the wrapping being different when not rendering the entire string. Instead, calculate the exact caret position when rending a text container and store the caret position in the text cursor for later use. IME usage guide (Windows) ------------------------------ 1. Install the respective language and the Microsoft-IME keyboard (e.g. for Chinese, Japanese or Korean). 2. Launch the game (or a text editor to first try out the IME). Note that Windows may track the input language separately for every application. You can change this in the Windows input settings so the input language is changed globally. 2. Switch the input language using the hotkey Windows+Space or another hotkey that you configured in the Windows input settings (Alt+Shift is the default, but you should consider disabling it, to avoid accidentally changing the input language while playing). 3. Switch from Latin/English input mode to the respective asian input mode. - Chinese: Use Ctrl+Space to switch between English and Chinese input mode. You can change this hotkey in the IME's settings. - Japanese: Use Ctrl+Space to switch between Alphanumeric and Hiragana/Katakana input mode. You can change this hotkey in the IME's settings. - Korean: Use Right Alt to switch between English and Hangul input mode. You cannot change this hotkey as of yet. - Note that the input mode is also tracked per application, but there is no setting to change this behavior as far as I know, so you'll need to switch for every application separately. 4. Start typing. The underlined text is the current composition text. While a composition is active, you can only edit the composition text. Confirm the composition with Space or by selecting a candidate from the candidate list with the arrow keys. Cancel the composition with Escape or by using Backspace to delete the composition text. Note that not all languages offer a candidate list. SDL version-specific issues ------------------------------ - 2.26.5, 2.24.2, 2.0.22: IME candidates work. But there are minor bugs when moving the composition cursor. - 2.0.18, 2.0.20: IME candidates work. - 2.0.16 (our current version): IME candidates cannot be determined with Windows API. Windows tries to draw the composition window like before, so this does not work in fullscreen mode. - 2.0.8 (upstream 0.7): IME candidates work. But this SDL version is too old for us.
2023-01-03 21:28:38 +00:00
size_t Size, Count;
add str_utf8_stats to base system
2021-11-24 22:24:46 +00:00
str_utf8_stats("abc", 4, 3, &Size, &Count);
EXPECT_EQ(Size, 3);
EXPECT_EQ(Count, 3);
str_utf8_stats("abc", 2, 3, &Size, &Count);
EXPECT_EQ(Size, 1);
EXPECT_EQ(Count, 1);
str_utf8_stats("", 1, 0, &Size, &Count);
EXPECT_EQ(Size, 0);
EXPECT_EQ(Count, 0);
str_utf8_stats("abcde", 6, 5, &Size, &Count);
EXPECT_EQ(Size, 5);
EXPECT_EQ(Count, 5);
str_utf8_stats("любовь", 13, 6, &Size, &Count);
EXPECT_EQ(Size, 12);
EXPECT_EQ(Count, 6);
str_utf8_stats("abc愛", 7, 4, &Size, &Count);
EXPECT_EQ(Size, 6);
EXPECT_EQ(Count, 4);
str_utf8_stats("abc愛", 6, 4, &Size, &Count);
EXPECT_EQ(Size, 3);
EXPECT_EQ(Count, 3);
str_utf8_stats("любовь", 13, 3, &Size, &Count);
EXPECT_EQ(Size, 6);
EXPECT_EQ(Count, 3);
}
Add utility functions for converting UTF-8 bytes ↔ chars offsets Add `str_utf8_offset_bytes_to_chars` and `str_utf8_offset_chars_to_bytes` functions to base system to convert between byte and UTF-8 character offsets in UTF-8 strings. Previously, this was separately implemented in the textrender and in the lineinput helper. These textrender functions are entirely replaced by the new functions: - `ITextRender::SelectionToUTF8OffSets` (by `str_utf8_offset_chars_to_bytes`) - `ITextRender::UTF8OffToDecodedOff` (by `str_utf8_offset_bytes_to_chars`) - `ITextRender::DecodedOffToUTF8Off` (by `str_utf8_offset_chars_to_bytes`) These lineinput helper functions are reimplemented using the new functions: - `CLineInput::OffsetFromActualToDisplay` (uses `str_utf8_offset_bytes_to_chars`) - `CLineInput::OffsetFromDisplayToActual` (uses `str_utf8_offset_chars_to_bytes`)
2023-08-15 19:32:56 +00:00
TEST(Str, Utf8OffsetBytesToChars)
{
EXPECT_EQ(str_utf8_offset_bytes_to_chars("", 0), 0);
EXPECT_EQ(str_utf8_offset_bytes_to_chars("", 100), 0);
EXPECT_EQ(str_utf8_offset_bytes_to_chars("abc", 0), 0);
EXPECT_EQ(str_utf8_offset_bytes_to_chars("abc", 1), 1);
EXPECT_EQ(str_utf8_offset_bytes_to_chars("abc", 2), 2);
EXPECT_EQ(str_utf8_offset_bytes_to_chars("abc", 3), 3);
EXPECT_EQ(str_utf8_offset_bytes_to_chars("abc", 100), 3);
EXPECT_EQ(str_utf8_offset_bytes_to_chars("любовь", 0), 0);
EXPECT_EQ(str_utf8_offset_bytes_to_chars("любовь", 2), 1);
EXPECT_EQ(str_utf8_offset_bytes_to_chars("любовь", 4), 2);
EXPECT_EQ(str_utf8_offset_bytes_to_chars("любовь", 6), 3);
EXPECT_EQ(str_utf8_offset_bytes_to_chars("любовь", 8), 4);
EXPECT_EQ(str_utf8_offset_bytes_to_chars("любовь", 10), 5);
EXPECT_EQ(str_utf8_offset_bytes_to_chars("любовь", 12), 6);
EXPECT_EQ(str_utf8_offset_bytes_to_chars("любовь", 100), 6);
EXPECT_EQ(str_utf8_offset_bytes_to_chars("DDNet最好了", 5), 5);
EXPECT_EQ(str_utf8_offset_bytes_to_chars("DDNet最好了", 8), 6);
EXPECT_EQ(str_utf8_offset_bytes_to_chars("DDNet最好了", 11), 7);
EXPECT_EQ(str_utf8_offset_bytes_to_chars("DDNet最好了", 14), 8);
EXPECT_EQ(str_utf8_offset_bytes_to_chars("DDNet最好了", 100), 8);
}
TEST(Str, Utf8OffsetCharsToBytes)
{
EXPECT_EQ(str_utf8_offset_chars_to_bytes("", 0), 0);
EXPECT_EQ(str_utf8_offset_chars_to_bytes("", 100), 0);
EXPECT_EQ(str_utf8_offset_chars_to_bytes("abc", 0), 0);
EXPECT_EQ(str_utf8_offset_chars_to_bytes("abc", 1), 1);
EXPECT_EQ(str_utf8_offset_chars_to_bytes("abc", 2), 2);
EXPECT_EQ(str_utf8_offset_chars_to_bytes("abc", 3), 3);
EXPECT_EQ(str_utf8_offset_chars_to_bytes("abc", 100), 3);
EXPECT_EQ(str_utf8_offset_chars_to_bytes("любовь", 0), 0);
EXPECT_EQ(str_utf8_offset_chars_to_bytes("любовь", 1), 2);
EXPECT_EQ(str_utf8_offset_chars_to_bytes("любовь", 2), 4);
EXPECT_EQ(str_utf8_offset_chars_to_bytes("любовь", 3), 6);
EXPECT_EQ(str_utf8_offset_chars_to_bytes("любовь", 4), 8);
EXPECT_EQ(str_utf8_offset_chars_to_bytes("любовь", 5), 10);
EXPECT_EQ(str_utf8_offset_chars_to_bytes("любовь", 6), 12);
EXPECT_EQ(str_utf8_offset_chars_to_bytes("любовь", 100), 12);
EXPECT_EQ(str_utf8_offset_chars_to_bytes("DDNet最好了", 5), 5);
EXPECT_EQ(str_utf8_offset_chars_to_bytes("DDNet最好了", 6), 8);
EXPECT_EQ(str_utf8_offset_chars_to_bytes("DDNet最好了", 7), 11);
EXPECT_EQ(str_utf8_offset_chars_to_bytes("DDNet最好了", 8), 14);
EXPECT_EQ(str_utf8_offset_chars_to_bytes("DDNet最好了", 100), 14);
}
Unify string test naming
2022-09-11 21:40:32 +00:00
TEST(Str, Time)
Add str_time/str_time_float Handles centisecs < 0 Uses int64, otherwise we're limited to ~260 days
2020-10-18 20:44:02 +00:00
{
char aBuf[32] = "foobar";
EXPECT_EQ(str_time(123456, TIME_DAYS, aBuf, 0), -1);
EXPECT_STREQ(aBuf, "foobar");
Add finish info messages.
2023-11-20 11:23:56 +00:00
EXPECT_EQ(str_time(123456, TIME_SECS_CENTISECS + 1, aBuf, sizeof(aBuf)), -1);
Add str_time/str_time_float Handles centisecs < 0 Uses int64, otherwise we're limited to ~260 days
2020-10-18 20:44:02 +00:00
EXPECT_STREQ(aBuf, "");
Add finish info messages.
2023-11-20 11:23:56 +00:00
EXPECT_EQ(str_time(-123456, TIME_MINS_CENTISECS, aBuf, sizeof(aBuf)), 5);
EXPECT_STREQ(aBuf, "00.00");
Add str_time/str_time_float Handles centisecs < 0 Uses int64, otherwise we're limited to ~260 days
2020-10-18 20:44:02 +00:00
EXPECT_EQ(str_time(INT64_MAX, TIME_DAYS, aBuf, sizeof(aBuf)), 23);
EXPECT_STREQ(aBuf, "1067519911673d 00:09:18");
EXPECT_EQ(str_time(123456, TIME_DAYS, aBuf, sizeof(aBuf)), 5);
EXPECT_STREQ(aBuf, "20:34");
EXPECT_EQ(str_time(1234567, TIME_DAYS, aBuf, sizeof(aBuf)), 8);
EXPECT_STREQ(aBuf, "03:25:45");
EXPECT_EQ(str_time(12345678, TIME_DAYS, aBuf, sizeof(aBuf)), 11);
EXPECT_STREQ(aBuf, "1d 10:17:36");
EXPECT_EQ(str_time(123456, TIME_HOURS, aBuf, sizeof(aBuf)), 5);
EXPECT_STREQ(aBuf, "20:34");
EXPECT_EQ(str_time(1234567, TIME_HOURS, aBuf, sizeof(aBuf)), 8);
EXPECT_STREQ(aBuf, "03:25:45");
EXPECT_EQ(str_time(12345678, TIME_HOURS, aBuf, sizeof(aBuf)), 8);
EXPECT_STREQ(aBuf, "34:17:36");
EXPECT_EQ(str_time(123456, TIME_MINS, aBuf, sizeof(aBuf)), 5);
EXPECT_STREQ(aBuf, "20:34");
EXPECT_EQ(str_time(1234567, TIME_MINS, aBuf, sizeof(aBuf)), 6);
EXPECT_STREQ(aBuf, "205:45");
EXPECT_EQ(str_time(12345678, TIME_MINS, aBuf, sizeof(aBuf)), 7);
EXPECT_STREQ(aBuf, "2057:36");
EXPECT_EQ(str_time(123456, TIME_HOURS_CENTISECS, aBuf, sizeof(aBuf)), 8);
EXPECT_STREQ(aBuf, "20:34.56");
EXPECT_EQ(str_time(1234567, TIME_HOURS_CENTISECS, aBuf, sizeof(aBuf)), 11);
EXPECT_STREQ(aBuf, "03:25:45.67");
EXPECT_EQ(str_time(12345678, TIME_HOURS_CENTISECS, aBuf, sizeof(aBuf)), 11);
EXPECT_STREQ(aBuf, "34:17:36.78");
EXPECT_EQ(str_time(123456, TIME_MINS_CENTISECS, aBuf, sizeof(aBuf)), 8);
EXPECT_STREQ(aBuf, "20:34.56");
EXPECT_EQ(str_time(1234567, TIME_MINS_CENTISECS, aBuf, sizeof(aBuf)), 9);
EXPECT_STREQ(aBuf, "205:45.67");
EXPECT_EQ(str_time(12345678, TIME_MINS_CENTISECS, aBuf, sizeof(aBuf)), 10);
EXPECT_STREQ(aBuf, "2057:36.78");
Add finish info messages.
2023-11-20 11:23:56 +00:00
EXPECT_EQ(str_time(123456, TIME_SECS_CENTISECS, aBuf, sizeof(aBuf)), 5);
EXPECT_STREQ(aBuf, "34.56");
EXPECT_EQ(str_time(1234567, TIME_SECS_CENTISECS, aBuf, sizeof(aBuf)), 5);
EXPECT_STREQ(aBuf, "45.67");
EXPECT_EQ(str_time(12345678, TIME_SECS_CENTISECS, aBuf, sizeof(aBuf)), 5);
EXPECT_STREQ(aBuf, "36.78");
Add str_time/str_time_float Handles centisecs < 0 Uses int64, otherwise we're limited to ~260 days
2020-10-18 20:44:02 +00:00
}
Unify string test naming
2022-09-11 21:40:32 +00:00
TEST(Str, TimeFloat)
Add str_time/str_time_float Handles centisecs < 0 Uses int64, otherwise we're limited to ~260 days
2020-10-18 20:44:02 +00:00
{
char aBuf[64];
EXPECT_EQ(str_time_float(123456.78, TIME_DAYS, aBuf, sizeof(aBuf)), 11);
EXPECT_STREQ(aBuf, "1d 10:17:36");
Fix centisecs rounding in str_time_float
2021-01-08 10:21:37 +00:00
Add finish info messages.
2023-11-20 11:23:56 +00:00
EXPECT_EQ(str_time_float(12.16, TIME_HOURS_CENTISECS, aBuf, sizeof(aBuf)), 5);
EXPECT_STREQ(aBuf, "12.16");
Round by millisecond, not centisecond (fixes #6453) Some servers seem to allow times with millisecond precision, official DDNet doesn't, so never noticed this issue.
2023-03-23 08:46:50 +00:00
EXPECT_EQ(str_time_float(22.995, TIME_MINS, aBuf, sizeof(aBuf)), 5);
EXPECT_STREQ(aBuf, "00:22");
Add str_time/str_time_float Handles centisecs < 0 Uses int64, otherwise we're limited to ~260 days
2020-10-18 20:44:02 +00:00
}
Add `str_has_cc` to check for control characters in strings
2022-09-11 21:39:16 +00:00
TEST(Str, HasCc)
{
EXPECT_FALSE(str_has_cc(""));
EXPECT_FALSE(str_has_cc("a"));
EXPECT_FALSE(str_has_cc("Merhaba dünya!"));
EXPECT_TRUE(str_has_cc("\n"));
EXPECT_TRUE(str_has_cc("\r"));
EXPECT_TRUE(str_has_cc("\t"));
EXPECT_TRUE(str_has_cc("a\n"));
EXPECT_TRUE(str_has_cc("a\rb"));
EXPECT_TRUE(str_has_cc("\tb"));
EXPECT_TRUE(str_has_cc("\n\n"));
Extend tests of `str_has_cc` with other control characters The character `\n` was checked redundantly. Now more obscure control characters are also checked instead.
2022-09-28 22:10:39 +00:00
EXPECT_TRUE(str_has_cc("\x1C"));
EXPECT_TRUE(str_has_cc("\x1D"));
EXPECT_TRUE(str_has_cc("\x1E"));
EXPECT_TRUE(str_has_cc("\x1F"));
Add `str_has_cc` to check for control characters in strings
2022-09-11 21:39:16 +00:00
}
Sort filenames case insensitive with `str_comp_filenames`, add tests This changes `str_comp_filenames` so it sorts filenames case insensitive while also comparing digits characters as numbers. This makes filename sorting consistent with the behavior in Windows Explorer.
2022-09-28 22:00:37 +00:00
Add tests for `str_sanitize_cc`
2022-09-28 22:11:05 +00:00
TEST(Str, SanitizeCc)
{
char aBuf[64];
str_copy(aBuf, "");
str_sanitize_cc(aBuf);
EXPECT_STREQ(aBuf, "");
str_copy(aBuf, "a");
str_sanitize_cc(aBuf);
EXPECT_STREQ(aBuf, "a");
str_copy(aBuf, "Merhaba dünya!");
str_sanitize_cc(aBuf);
EXPECT_STREQ(aBuf, "Merhaba dünya!");
str_copy(aBuf, "\n");
str_sanitize_cc(aBuf);
EXPECT_STREQ(aBuf, " ");
str_copy(aBuf, "\r");
str_sanitize_cc(aBuf);
EXPECT_STREQ(aBuf, " ");
str_copy(aBuf, "\t");
str_sanitize_cc(aBuf);
EXPECT_STREQ(aBuf, " ");
str_copy(aBuf, "a\n");
str_sanitize_cc(aBuf);
EXPECT_STREQ(aBuf, "a ");
str_copy(aBuf, "a\rb");
str_sanitize_cc(aBuf);
EXPECT_STREQ(aBuf, "a b");
str_copy(aBuf, "\tb");
str_sanitize_cc(aBuf);
EXPECT_STREQ(aBuf, " b");
str_copy(aBuf, "\n\n");
str_sanitize_cc(aBuf);
EXPECT_STREQ(aBuf, " ");
str_copy(aBuf, "\x1C");
str_sanitize_cc(aBuf);
EXPECT_STREQ(aBuf, " ");
str_copy(aBuf, "\x1D");
str_sanitize_cc(aBuf);
EXPECT_STREQ(aBuf, " ");
str_copy(aBuf, "\x1E");
str_sanitize_cc(aBuf);
EXPECT_STREQ(aBuf, " ");
str_copy(aBuf, "\x1F");
str_sanitize_cc(aBuf);
EXPECT_STREQ(aBuf, " ");
}
Add tests for `str_sanitize`
2022-09-28 22:15:53 +00:00
TEST(Str, Sanitize)
{
char aBuf[64];
str_copy(aBuf, "");
str_sanitize(aBuf);
EXPECT_STREQ(aBuf, "");
str_copy(aBuf, "a");
str_sanitize(aBuf);
EXPECT_STREQ(aBuf, "a");
str_copy(aBuf, "Merhaba dünya!");
str_sanitize(aBuf);
EXPECT_STREQ(aBuf, "Merhaba dünya!");
str_copy(aBuf, "\n");
str_sanitize(aBuf);
EXPECT_STREQ(aBuf, "\n");
str_copy(aBuf, "\r");
str_sanitize(aBuf);
EXPECT_STREQ(aBuf, "\r");
str_copy(aBuf, "\t");
str_sanitize(aBuf);
EXPECT_STREQ(aBuf, "\t");
str_copy(aBuf, "a\n");
str_sanitize(aBuf);
EXPECT_STREQ(aBuf, "a\n");
str_copy(aBuf, "a\rb");
str_sanitize(aBuf);
EXPECT_STREQ(aBuf, "a\rb");
str_copy(aBuf, "\tb");
str_sanitize(aBuf);
EXPECT_STREQ(aBuf, "\tb");
str_copy(aBuf, "\n\n");
str_sanitize(aBuf);
EXPECT_STREQ(aBuf, "\n\n");
str_copy(aBuf, "\x1C");
str_sanitize(aBuf);
EXPECT_STREQ(aBuf, " ");
str_copy(aBuf, "\x1D");
str_sanitize(aBuf);
EXPECT_STREQ(aBuf, " ");
str_copy(aBuf, "\x1E");
str_sanitize(aBuf);
EXPECT_STREQ(aBuf, " ");
str_copy(aBuf, "\x1F");
str_sanitize(aBuf);
EXPECT_STREQ(aBuf, " ");
}
Add tests for `str_clean_whitespaces`, fix documentation
2022-09-28 21:29:55 +00:00
TEST(Str, CleanWhitespaces)
{
char aBuf[64];
str_copy(aBuf, "aa bb ccc dddd eeeee");
str_clean_whitespaces(aBuf);
EXPECT_STREQ(aBuf, "aa bb ccc dddd eeeee");
str_copy(aBuf, " ");
str_clean_whitespaces(aBuf);
EXPECT_STREQ(aBuf, "");
str_copy(aBuf, " aa");
str_clean_whitespaces(aBuf);
EXPECT_STREQ(aBuf, "aa");
str_copy(aBuf, "aa ");
str_clean_whitespaces(aBuf);
EXPECT_STREQ(aBuf, "aa");
str_copy(aBuf, " aa bb ccc dddd eeeee ");
str_clean_whitespaces(aBuf);
EXPECT_STREQ(aBuf, "aa bb ccc dddd eeeee");
}
Use `str_isspace` in `str_skip_(to_)whitespace(s)`, add tests Update the `str_skip_to_whitespace(_const)` functions according to their documentation, which already stated that `\r` was also considered as whitespace.
2022-09-28 14:41:03 +00:00
TEST(Str, SkipToWhitespace)
{
char aBuf[64];
str_copy(aBuf, "");
EXPECT_EQ(str_skip_to_whitespace(aBuf), aBuf);
EXPECT_EQ(str_skip_to_whitespace_const(aBuf), aBuf);
str_copy(aBuf, " a");
EXPECT_EQ(str_skip_to_whitespace(aBuf), aBuf);
EXPECT_EQ(str_skip_to_whitespace_const(aBuf), aBuf);
str_copy(aBuf, "aaaa b");
EXPECT_EQ(str_skip_to_whitespace(aBuf), aBuf + 4);
EXPECT_EQ(str_skip_to_whitespace_const(aBuf), aBuf + 4);
str_copy(aBuf, "aaaa\n\nb");
EXPECT_EQ(str_skip_to_whitespace(aBuf), aBuf + 4);
EXPECT_EQ(str_skip_to_whitespace_const(aBuf), aBuf + 4);
str_copy(aBuf, "aaaa\r\rb");
EXPECT_EQ(str_skip_to_whitespace(aBuf), aBuf + 4);
EXPECT_EQ(str_skip_to_whitespace_const(aBuf), aBuf + 4);
str_copy(aBuf, "aaaa\t\tb");
EXPECT_EQ(str_skip_to_whitespace(aBuf), aBuf + 4);
EXPECT_EQ(str_skip_to_whitespace_const(aBuf), aBuf + 4);
}
TEST(Str, SkipWhitespaces)
{
char aBuf[64];
str_copy(aBuf, "");
EXPECT_EQ(str_skip_whitespaces(aBuf), aBuf);
EXPECT_EQ(str_skip_whitespaces_const(aBuf), aBuf);
str_copy(aBuf, "aaaa");
EXPECT_EQ(str_skip_whitespaces(aBuf), aBuf);
EXPECT_EQ(str_skip_whitespaces_const(aBuf), aBuf);
str_copy(aBuf, " \n\r\taaaa");
EXPECT_EQ(str_skip_whitespaces(aBuf), aBuf + 4);
EXPECT_EQ(str_skip_whitespaces_const(aBuf), aBuf + 4);
}
Sort filenames case insensitive with `str_comp_filenames`, add tests This changes `str_comp_filenames` so it sorts filenames case insensitive while also comparing digits characters as numbers. This makes filename sorting consistent with the behavior in Windows Explorer.
2022-09-28 22:00:37 +00:00
TEST(Str, CompFilename)
{
EXPECT_EQ(str_comp_filenames("a", "a"), 0);
EXPECT_LT(str_comp_filenames("a", "b"), 0);
EXPECT_GT(str_comp_filenames("b", "a"), 0);
EXPECT_EQ(str_comp_filenames("A", "a"), 0);
EXPECT_LT(str_comp_filenames("A", "b"), 0);
EXPECT_GT(str_comp_filenames("b", "A"), 0);
EXPECT_LT(str_comp_filenames("a", "B"), 0);
EXPECT_GT(str_comp_filenames("B", "a"), 0);
Added some test cases that were failing under new version
2022-12-30 00:11:57 +00:00
EXPECT_EQ(str_comp_filenames("1A", "1a"), 0);
EXPECT_LT(str_comp_filenames("1a", "1B"), 0);
EXPECT_GT(str_comp_filenames("1B", "1a"), 0);
EXPECT_LT(str_comp_filenames("1a", "1b"), 0);
EXPECT_GT(str_comp_filenames("1b", "1a"), 0);
EXPECT_GT(str_comp_filenames("12a", "1B"), 0);
EXPECT_LT(str_comp_filenames("1B", "12a"), 0);
Add some more test to str_comp_filenames and restore old behavior
2023-01-02 20:20:49 +00:00
EXPECT_GT(str_comp_filenames("10a", "1B"), 0);
EXPECT_LT(str_comp_filenames("1B", "10a"), 0);
EXPECT_GT(str_comp_filenames("10a", "00B"), 0);
EXPECT_LT(str_comp_filenames("00B", "10a"), 0);
EXPECT_GT(str_comp_filenames("10a", "09B"), 0);
EXPECT_LT(str_comp_filenames("09B", "10a"), 0);
Sort filenames case insensitive with `str_comp_filenames`, add tests This changes `str_comp_filenames` so it sorts filenames case insensitive while also comparing digits characters as numbers. This makes filename sorting consistent with the behavior in Windows Explorer.
2022-09-28 22:00:37 +00:00
EXPECT_LT(str_comp_filenames("abc", "abcd"), 0);
EXPECT_GT(str_comp_filenames("abcd", "abc"), 0);
EXPECT_LT(str_comp_filenames("abc2", "abcd1"), 0);
EXPECT_GT(str_comp_filenames("abcd1", "abc2"), 0);
EXPECT_LT(str_comp_filenames("abc50", "abcd"), 0);
EXPECT_GT(str_comp_filenames("abcd", "abc50"), 0);
EXPECT_EQ(str_comp_filenames("file0", "file0"), 0);
EXPECT_LT(str_comp_filenames("file0", "file1"), 0);
EXPECT_GT(str_comp_filenames("file1", "file0"), 0);
Added some test cases that were failing under new version
2022-12-30 00:11:57 +00:00
EXPECT_LT(str_comp_filenames("file1", "file09"), 0);
EXPECT_GT(str_comp_filenames("file09", "file1"), 0);
EXPECT_LT(str_comp_filenames("file1", "file009"), 0);
EXPECT_GT(str_comp_filenames("file009", "file1"), 0);
Add some more test to str_comp_filenames and restore old behavior
2023-01-02 20:20:49 +00:00
EXPECT_GT(str_comp_filenames("file10", "file00"), 0);
EXPECT_LT(str_comp_filenames("file00", "file10"), 0);
EXPECT_GT(str_comp_filenames("file10", "file09"), 0);
EXPECT_LT(str_comp_filenames("file09", "file10"), 0);
Sort filenames case insensitive with `str_comp_filenames`, add tests This changes `str_comp_filenames` so it sorts filenames case insensitive while also comparing digits characters as numbers. This makes filename sorting consistent with the behavior in Windows Explorer.
2022-09-28 22:00:37 +00:00
EXPECT_LT(str_comp_filenames("file13", "file37"), 0);
EXPECT_GT(str_comp_filenames("file37", "file13"), 0);
Added some test cases that were failing under new version
2022-12-30 00:11:57 +00:00
EXPECT_LT(str_comp_filenames("file1.ext", "file09.ext"), 0);
EXPECT_GT(str_comp_filenames("file09.ext", "file1.ext"), 0);
EXPECT_LT(str_comp_filenames("file1.ext", "file009.ext"), 0);
EXPECT_GT(str_comp_filenames("file009.ext", "file1.ext"), 0);
EXPECT_EQ(str_comp_filenames("file0.ext", "file0.ext"), 0);
Sort filenames case insensitive with `str_comp_filenames`, add tests This changes `str_comp_filenames` so it sorts filenames case insensitive while also comparing digits characters as numbers. This makes filename sorting consistent with the behavior in Windows Explorer.
2022-09-28 22:00:37 +00:00
EXPECT_LT(str_comp_filenames("file13.ext", "file37.ext"), 0);
EXPECT_GT(str_comp_filenames("file37.ext", "file13.ext"), 0);
EXPECT_LT(str_comp_filenames("FILE13.EXT", "file37.ext"), 0);
EXPECT_GT(str_comp_filenames("file37.ext", "FILE13.EXT"), 0);
Add some more test to str_comp_filenames and restore old behavior
2023-01-02 20:20:49 +00:00
EXPECT_GT(str_comp_filenames("file10.ext", "file00.ext"), 0);
EXPECT_LT(str_comp_filenames("file00.ext", "file10.ext"), 0);
EXPECT_GT(str_comp_filenames("file10.ext", "file09.ext"), 0);
EXPECT_LT(str_comp_filenames("file09.ext", "file10.ext"), 0);
Sort filenames case insensitive with `str_comp_filenames`, add tests This changes `str_comp_filenames` so it sorts filenames case insensitive while also comparing digits characters as numbers. This makes filename sorting consistent with the behavior in Windows Explorer.
2022-09-28 22:00:37 +00:00
EXPECT_LT(str_comp_filenames("file42", "file1337"), 0);
EXPECT_GT(str_comp_filenames("file1337", "file42"), 0);
EXPECT_LT(str_comp_filenames("file42.ext", "file1337.ext"), 0);
EXPECT_GT(str_comp_filenames("file1337.ext", "file42.ext"), 0);
EXPECT_GT(str_comp_filenames("file4414520", "file2055"), 0);
EXPECT_LT(str_comp_filenames("file4414520", "file205523151812419"), 0);
}
Add tests for `str_rchr`, update documentation
2022-09-29 08:33:39 +00:00
TEST(Str, RightChar)
{
const char *pStr = "a bb ccc dddd eeeee";
EXPECT_EQ(str_rchr(pStr, 'a'), pStr);
EXPECT_EQ(str_rchr(pStr, 'b'), pStr + 3);
EXPECT_EQ(str_rchr(pStr, 'c'), pStr + 7);
EXPECT_EQ(str_rchr(pStr, 'd'), pStr + 12);
EXPECT_EQ(str_rchr(pStr, ' '), pStr + 19);
EXPECT_EQ(str_rchr(pStr, 'e'), pStr + 24);
EXPECT_EQ(str_rchr(pStr, '\0'), pStr + str_length(pStr));
EXPECT_EQ(str_rchr(pStr, 'y'), nullptr);
}
Add tests for `str_countchr`, update documentation
2022-09-29 08:41:22 +00:00
TEST(Str, CountChar)
{
const char *pStr = "a bb ccc dddd eeeee";
EXPECT_EQ(str_countchr(pStr, 'a'), 1);
EXPECT_EQ(str_countchr(pStr, 'b'), 2);
EXPECT_EQ(str_countchr(pStr, 'c'), 3);
EXPECT_EQ(str_countchr(pStr, 'd'), 4);
EXPECT_EQ(str_countchr(pStr, 'e'), 5);
EXPECT_EQ(str_countchr(pStr, ' '), 10);
EXPECT_EQ(str_countchr(pStr, '\0'), 0);
EXPECT_EQ(str_countchr(pStr, 'y'), 0);
}
Add `windows_utf8_to_wide` and `windows_wide_to_utf8` To convert between UTF-8 encoded strings and wide character strings used with the Windows API. The functions use `std::wstring` and `std::string`, to simplify the usage and support arguments with arbitrary length. Also add some tests for the functions.
2023-05-02 15:36:01 +00:00
Add validation for `StrToInts` and `IntsToStr` Add strict validation for `StrToInts` function. Because this function should only be used with trusted internal strings, assertions are added to ensure that the string is not truncated. Some buffer sizes are adjusted accordingly, so truncation cannot happen. Add less strict validation for `IntsToStr` function. An additional argument specifying the size of the output buffer is added to assert that the size of the output buffer is sufficient. However, because this function is used to decode data sent by the server and read from maps and ghosts, invalid input data should never result in crashes or invalid UTF-8 strings. The function will now unpack an empty string and return `false`, if the string contains invalid UTF-8. The inline definition of the functions is not wanted, because it requires adding a `system.h` include in `gamecore.h`. Therefore the tools now have to depend on game-shared, which previously only worked because the functions were inline. Tests are added to ensure the function still behaves the same as before for valid inputs and correctly handles invalid inputs.
2024-03-04 17:05:12 +00:00
TEST(Str, StrToInts)
{
int aInts[8];
StrToInts(aInts, 1, "a");
EXPECT_EQ(aInts[0], 0xE1808000);
StrToInts(aInts, 1, "ab");
EXPECT_EQ(aInts[0], 0xE1E28000);
StrToInts(aInts, 1, "abc");
EXPECT_EQ(aInts[0], 0xE1E2E300);
StrToInts(aInts, 2, "abcd");
EXPECT_EQ(aInts[0], 0xE1E2E3E4);
EXPECT_EQ(aInts[1], 0x80808000);
StrToInts(aInts, 2, "abcde");
EXPECT_EQ(aInts[0], 0xE1E2E3E4);
EXPECT_EQ(aInts[1], 0xE5808000);
StrToInts(aInts, 2, "abcdef");
EXPECT_EQ(aInts[0], 0xE1E2E3E4);
EXPECT_EQ(aInts[1], 0xE5E68000);
StrToInts(aInts, 2, "abcdefg");
EXPECT_EQ(aInts[0], 0xE1E2E3E4);
EXPECT_EQ(aInts[1], 0xE5E6E700);
StrToInts(aInts, 2, "öüä");
EXPECT_EQ(aInts[0], 0x4336433C);
EXPECT_EQ(aInts[1], 0x43248000);
StrToInts(aInts, 3, "aβい🐘");
EXPECT_EQ(aInts[0], 0xE14E3263);
EXPECT_EQ(aInts[1], 0x0104701F);
EXPECT_EQ(aInts[2], 0x10188000);
// long padding
StrToInts(aInts, 4, "abc");
EXPECT_EQ(aInts[0], 0xE1E2E380);
EXPECT_EQ(aInts[1], 0x80808080);
EXPECT_EQ(aInts[2], 0x80808080);
EXPECT_EQ(aInts[3], 0x80808000);
}
TEST(Str, IntsToStr)
{
int aInts[8];
char aStr[sizeof(aInts)];
aInts[0] = 0xE1808000;
EXPECT_TRUE(IntsToStr(aInts, 1, aStr, std::size(aStr)));
EXPECT_STREQ(aStr, "a");
aInts[0] = 0xE1E28000;
EXPECT_TRUE(IntsToStr(aInts, 1, aStr, std::size(aStr)));
EXPECT_STREQ(aStr, "ab");
aInts[0] = 0xE1E2E300;
EXPECT_TRUE(IntsToStr(aInts, 1, aStr, std::size(aStr)));
EXPECT_STREQ(aStr, "abc");
aInts[0] = 0xE1E2E3E4;
aInts[1] = 0x80808000;
EXPECT_TRUE(IntsToStr(aInts, 2, aStr, std::size(aStr)));
EXPECT_STREQ(aStr, "abcd");
aInts[0] = 0xE1E2E3E4;
aInts[1] = 0xE5808000;
EXPECT_TRUE(IntsToStr(aInts, 2, aStr, std::size(aStr)));
EXPECT_STREQ(aStr, "abcde");
aInts[0] = 0xE1E2E3E4;
aInts[1] = 0xE5E68000;
EXPECT_TRUE(IntsToStr(aInts, 2, aStr, std::size(aStr)));
EXPECT_STREQ(aStr, "abcdef");
aInts[0] = 0xE1E2E3E4;
aInts[1] = 0xE5E6E700;
EXPECT_TRUE(IntsToStr(aInts, 2, aStr, std::size(aStr)));
EXPECT_STREQ(aStr, "abcdefg");
aInts[0] = 0x4336433C;
aInts[1] = 0x43248000;
EXPECT_TRUE(IntsToStr(aInts, 2, aStr, std::size(aStr)));
EXPECT_STREQ(aStr, "öüä");
aInts[0] = 0xE14E3263;
aInts[1] = 0x0104701F;
aInts[2] = 0x10188000;
EXPECT_TRUE(IntsToStr(aInts, 3, aStr, std::size(aStr)));
EXPECT_STREQ(aStr, "aβい🐘");
// long padding
aInts[0] = 0xE1E2E380;
aInts[1] = 0x80808080;
aInts[2] = 0x80808080;
aInts[3] = 0x80808000;
EXPECT_TRUE(IntsToStr(aInts, 4, aStr, std::size(aStr)));
EXPECT_STREQ(aStr, "abc");
// early null character (0x80)
aInts[0] = 0xE1E2E380;
aInts[1] = 0xE1E2E3E4;
aInts[2] = 0xE1E2E3E4;
aInts[3] = 0xE1E2E300;
EXPECT_TRUE(IntsToStr(aInts, 4, aStr, std::size(aStr)));
EXPECT_STREQ(aStr, "abc");
// invalid UTF-8
aInts[0] = 0xE17FE200;
EXPECT_FALSE(IntsToStr(aInts, 1, aStr, std::size(aStr)));
EXPECT_STREQ(aStr, "");
// invalid UTF-8 (0x00 in string data, which is translated to '\x80')
aInts[0] = 0xE100E200;
EXPECT_FALSE(IntsToStr(aInts, 1, aStr, std::size(aStr)));
EXPECT_STREQ(aStr, "");
// invalid UTF-8
aInts[0] = 0xE1E2E36F;
aInts[1] = 0x3F40E4E5;
aInts[2] = 0xE67FE700;
EXPECT_FALSE(IntsToStr(aInts, 3, aStr, std::size(aStr)));
EXPECT_STREQ(aStr, "");
// invalid UTF-8 and missing null-terminator
aInts[0] = 0x7F7F7F7F;
EXPECT_FALSE(IntsToStr(aInts, 1, aStr, std::size(aStr)));
EXPECT_STREQ(aStr, "");
// missing null-terminator at the end is ignored
aInts[0] = 0xE1E2E3E4;
EXPECT_TRUE(IntsToStr(aInts, 1, aStr, std::size(aStr)));
EXPECT_STREQ(aStr, "abc");
// basic fuzzing: no input integer should result in invalid UTF-8 in the string
for(int i = 0; i <= 0xFFFFFF; i += 7) // checking all values takes a bit too long
{
mem_zero(aStr, std::size(aStr));
aInts[0] = 0xE1 << 24 | i;
aInts[1] = i << 8 | 0xE2;
aInts[2] = 0xE3 << 24 | i;
const bool ConversionResult = IntsToStr(aInts, 3, aStr, std::size(aStr));
// ensure null-termination before calling str_utf8_check
ASSERT_TRUE(mem_has_null(aStr, std::size(aStr)));
ASSERT_TRUE(str_utf8_check(aStr));
ASSERT_TRUE(ConversionResult || aStr[0] == '\0');
}
}
Add `windows_utf8_to_wide` and `windows_wide_to_utf8` To convert between UTF-8 encoded strings and wide character strings used with the Windows API. The functions use `std::wstring` and `std::string`, to simplify the usage and support arguments with arbitrary length. Also add some tests for the functions.
2023-05-02 15:36:01 +00:00
#if defined(CONF_FAMILY_WINDOWS)
TEST(Str, WindowsUtf8WideConversion)
{
const char *apUtf8Strings[] = {
"",
"abc",
"a bb ccc dddd eeeee",
"öüä",
"привет Наташа",
"ąçęįǫų",
"DDNet最好了",
"aβい🐘"};
const wchar_t *apWideStrings[] = {
L"",
L"abc",
L"a bb ccc dddd eeeee",
L"öüä",
L"привет Наташа",
L"ąçęįǫų",
L"DDNet最好了",
L"aβい🐘"};
static_assert(std::size(apUtf8Strings) == std::size(apWideStrings));
for(size_t i = 0; i < std::size(apUtf8Strings); i++)
{
Check for invalid unicode in filenames etc. on Windows Add stricter error handling when converting between UTF-16 (wide characters) and UTF-8 (multi-byte) on Windows. The `windows_wide_to_utf8` function now returns an `std::optional`, which will be empty if the argument contains invalid UTF-16. Files/folders with names containing invalid UTF-16 are now ignored on Windows. It was previously not possible to use these files either, as converting their names to UTF-8 changed the invalid codepoints to unicode replacement characters. The `windows_utf8_to_wide` function now fails with an assertion error if the argument contains invalid UTF-8, as this should never happen. Closes #7486.
2023-12-01 19:04:48 +00:00
const std::optional<std::string> ConvertedUtf8 = windows_wide_to_utf8(apWideStrings[i]);
Add `windows_utf8_to_wide` and `windows_wide_to_utf8` To convert between UTF-8 encoded strings and wide character strings used with the Windows API. The functions use `std::wstring` and `std::string`, to simplify the usage and support arguments with arbitrary length. Also add some tests for the functions.
2023-05-02 15:36:01 +00:00
const std::wstring ConvertedWide = windows_utf8_to_wide(apUtf8Strings[i]);
Check for invalid unicode in filenames etc. on Windows Add stricter error handling when converting between UTF-16 (wide characters) and UTF-8 (multi-byte) on Windows. The `windows_wide_to_utf8` function now returns an `std::optional`, which will be empty if the argument contains invalid UTF-16. Files/folders with names containing invalid UTF-16 are now ignored on Windows. It was previously not possible to use these files either, as converting their names to UTF-8 changed the invalid codepoints to unicode replacement characters. The `windows_utf8_to_wide` function now fails with an assertion error if the argument contains invalid UTF-8, as this should never happen. Closes #7486.
2023-12-01 19:04:48 +00:00
ASSERT_TRUE(ConvertedUtf8.has_value());
EXPECT_STREQ(ConvertedUtf8.value().c_str(), apUtf8Strings[i]);
Add `windows_utf8_to_wide` and `windows_wide_to_utf8` To convert between UTF-8 encoded strings and wide character strings used with the Windows API. The functions use `std::wstring` and `std::string`, to simplify the usage and support arguments with arbitrary length. Also add some tests for the functions.
2023-05-02 15:36:01 +00:00
EXPECT_STREQ(ConvertedWide.c_str(), apWideStrings[i]);
}
}
#endif
Reference in a new issue Copy permalink