Commit graph

11 commits

Author SHA1 Message Date
0x4261756D
61f6c6c966 LibTextCodec: Add SingleByteEncoders
They are similar to their already existing decoder counterparts.

(cherry picked from commit 96de4ef7e00f44b3f7913db221625940da7f561a)
2024-10-16 10:25:48 -04:00
BenJilks
ab96bf642b LibTextCodec: Implement iso-2022-jp encoder
Implements the `iso-2022-jp` encoder, as specified by
https://encoding.spec.whatwg.org/#iso-2022-jp-encoder

(cherry picked from commit 0ca5675d59dbcb52cedea56729de26b41074024a)
2024-10-15 22:54:51 -04:00
BenJilks
9a4156482d LibTextCodec: Implement shift_jis encoder
Implements the `shift_jis` encoder, as specified by
https://encoding.spec.whatwg.org/#shift_jis-encoder

(cherry picked from commit 08a8d67a5bfa9690eb61977f75b2ed3dc73aa341)
2024-10-15 22:54:51 -04:00
BenJilks
1739838868 LibTextCodec: Implement gb18030 and gbk encoders
Implements the `gb18030` and `gbk` encoders, as specified by
https://encoding.spec.whatwg.org/#gb18030-encoder
https://encoding.spec.whatwg.org/#gbk-encoder

(cherry picked from commit d80575a4101ab0fbc22ff2b714c74530a965cd5c)
2024-10-15 22:54:51 -04:00
BenJilks
4cd250dc2d LibTextCodec: Implement big5 encoder
Implements the `big5` encoder, as specified by
https://encoding.spec.whatwg.org/#big5-encoder

(cherry picked from commit 34c8c559c112796af0f99b48a7b88cb26633a764)
2024-10-15 22:54:51 -04:00
BenJilks
399dc388d6 LibTextCodec: Implement euc-kr encoder
Implements the `euc-kr` encoder, as specified by
https://encoding.spec.whatwg.org/#euc-kr-encoder

(cherry picked from commit 826292536c0e6f82e7173a98f1f3b24216d82fec)
2024-10-15 22:54:51 -04:00
BenJilks
4167d1214a LibTextCodec+LibURL: Implement utf-8 and euc-jp encoders
Implements the corresponding encoders, selects the appropriate one when
encoding URL search params. If an encoder for the given encoding could
not be found, fallback to utf-8.

(cherry picked from commit 72d0e3284b604c4c1373fb019250cdf5bd492300)
2024-10-15 22:54:51 -04:00
Sam Atkins
2db168acc1 LibTextCodec+Everywhere: Port Decoders to new Strings 2023-02-19 17:15:47 +01:00
Nico Weber
3423b54eb9 LibTextCodec: Make utf-16be and utf-16le codecs actually work
There were two problems:

1. They didn't handle surrogates
2. They used signed chars, leading to eg 0x00e4 being treated as 0xffe4

Also add a basic test that catches both issues.
There's some code duplication with Utf16CodePointIterator::operator*(),
but let's get things working first.
2023-01-22 21:30:44 +00:00
sin-ack
3f3f45580a Everywhere: Add sv suffix to strings relying on StringView(char const*)
Each of these strings would previously rely on StringView's char const*
constructor overload, which would call __builtin_strlen on the string.
Since we now have operator ""sv, we can replace these with much simpler
versions. This opens the door to being able to remove
StringView(char const*).

No functional changes.
2022-07-12 23:11:35 +02:00
Karol Kosek
dcb24e943d Tests: Add a basic UTF-8 to UTF-8 LibTextCodec test 2022-03-29 01:01:32 +02:00