Commit graph

964 commits

Author SHA1 Message Date
Jan de Visser
224804b424 LibSQL: BTree index, Heap, and Meta objects for SQL Storage layer
Unfortunately this patch is quite large.

The main functionality included are a BTree index implementation and
the Heap class which manages persistent storage.

Also included are a Key subclass of the Tuple class, which is a
specialization for index key tuples. This "dragged in" the Meta layer,
which has classes defining SQL objects like tables and indexes.
2021-06-19 22:06:45 +02:00
Jan de Visser
2a46529170 LibSQL: Basic dynamic value classes for SQL Storage layer
This patch adds the basic dynamic value classes used by the SQL Storage
layer. The most elementary class is Value, which holds a typed Value
which can be converted to standard C++ types. A Tuple is a collection
of Values described by a TupleDescriptor, which specifies the names,
types, and ordering of the elements in the Tuple.

Tuples and Values can be serialized and deserialized to and from
ByteBuffers. This is mechanism which is used to save them to disk.

Tuples are used as keys in SQL indexes and rows in SQL tables.

Also included is a test file.
2021-06-19 22:06:45 +02:00
Idan Horowitz
5e53a690ac AK: Add support for keeping trailing zeros in fixed precision floats
This uses the same syntax as zero padding integers:
String::formatted("{:0.5}", 1.234) => "1.23400"
2021-06-19 16:13:59 +01:00
Peter Bocan
4d5ffd364a LibCrypto+LibTLS: Split and move test suite into Tests directory
This change splits test-crypto.cpp from Userland into separate test
suites located in Tests/ directory.
2021-06-19 19:05:36 +04:30
Kyle Ambroff-Kao
d173945dad Tests: Add a test for ProcFS fd interaction
Co-authored-by: Tim Schumacher <timschumi@gmx.de>
2021-06-18 10:15:14 +02:00
Ali Mohammad Pur
ac6a3d068d AK: Add some tests for hexdump formatting 2021-06-17 18:44:00 +04:30
Gunnar Beutner
631d36fd98 Everywhere: Add component declarations
This adds component declarations so that users can select to not build
certain parts of the OS.
2021-06-17 11:03:51 +02:00
sin-ack
9a9e7f03f2 Tests: Add test for case-insensitive matching 2021-06-16 16:30:12 +04:30
Brian Gianforcaro
b7f8343f87 AK+Tests: Add IntrusiveList<T,...>::insert_before(..) method
The insert_before method on AK::InlineLinkedList is used, so in order to
achieve feature parity, we need to implement it for AK::IntrusiveList as
well.
2021-06-16 10:40:01 +02:00
Idan Horowitz
8c7fe8d6c8 AK: Add support for removing SinglyLinkedList nodes during iteration
This commit also fixes the now-broken usage of SinglyLinkedList::remove
in the Piano application.
2021-06-15 23:59:21 +01:00
Jelle Raaijmakers
10e8b99038 LibC: Make getopt modify argv again
A POSIX-compatibility fix was introduced in 64740a0214 to make the
compilation of the `diffutils` port work, which expected a
`char* const* argv` signature.

And indeed, the POSIX spec does not mention permutation of `argv`:
https://pubs.opengroup.org/onlinepubs/9699919799/functions/getopt.html

However, most implementations do modify `argv` as evidenced by
documentation such as:
https://refspecs.linuxbase.org/LSB_5.0.0/LSB-Core-generic
    /LSB-Core-generic/libutil-getopt-3.html

  "The function prototype was aligned with POSIX 1003.1-2008 (ISO/IEC
   9945-2009) despite the fact that it modifies argv, and the library
   maintainers are unwilling to change this."

Change the behavior back to permutate `argc` to allow for the following
command line argument order to work again:

  unzip ./file.zip -o target-dir

Without this change, `./file.zip` in the example above would have been
ignored completely.
2021-06-13 17:05:03 +01:00
Andreas Kling
dc65f54c06 AK: Rename Vector::append(Vector) => Vector::extend(Vector)
Let's make it a bit more clear when we're appending the elements from
one vector to the end of another vector.
2021-06-12 13:24:45 +02:00
Idan Horowitz
77c2db4183 LibJS: Add all of the WeakMap.prototype methods (delete, get, has, set) 2021-06-12 10:44:28 +01:00
Hendiadyoin1
5ffe23e4f3 AK+LibX86: Generalize u128/256 to AK::UFixedBigInt
Doing these as custom classes might be faster, especially when writing
them in SSE, but this would cause a lot of Code duplication and due to
the nature of constexprs and the intelligence of the compiler they might
be using SSE/MMX either way
2021-06-11 18:14:11 +04:30
Idan Horowitz
a00d154522 LibJS: Notify WeakSets when heap cells are sweeped
This is an implementation of the following optional optimization:
https://tc39.es/ecma262/#sec-weakref-execution
2021-06-09 21:52:25 +01:00
Ali Mohammad Pur
3d94b5051d AK: Make Vector capable of holding reference types
This commit makes it possible to instantiate `Vector<T&>` and use it
to store references to `T` in a vector.
All non-pointer observers are made to return the reference, and the
pointer observers simply yield the underlying pointer.
Note that the 'find_*' methods act on the values and not the pointers
that are stored in the vector.
This commit also makes errors in various vector methods much more
readable by directly using requires-clauses on them.
And finally, it should be noted that Vector cannot hold temporaries :^)
2021-06-08 19:14:24 +02:00
Timothy Flynn
c7cd81bce8 LibSQL: Limit the number of nested subqueries
SQLite hasn't documented a limit on https://www.sqlite.org/limits.html
for the maximum number of nested subqueries. However, its parser is
generated with Yacc and has an internal limit of 100 for general nested
statements.

Fixes https://crbug.com/oss-fuzz/35022.
2021-06-08 19:08:13 +02:00
Timothy Flynn
4e974e6d60 LibSQL: Rename expression tree depth limit test case
Meant to rename this before committing the test - 'stack_limit' isn't a
great name when there's multiple test cases for various stack overflows.
2021-06-08 19:08:13 +02:00
Jelle Raaijmakers
d7126fbbc2 LibCore/ArgsParser: Learn how to stop on first non-option
We need this for utilities like `env`, that do not gain anything by
parsing the options passed to the command they are supposed to
execute.
2021-06-08 11:30:58 +02:00
Jelle Raaijmakers
0b0bce78f6 LibCore/ArgsParser: Add test suite
This adds a very basic test suite for ArgsParser that we can use to set
a baseline of functionality that we want to make sure keeps working.
2021-06-08 11:30:58 +02:00
Brian Gianforcaro
f5e04759cc AK: Add IntrusiveList::size_slow() to match InlineLinkedList
The functionality is needed to replace InlineLinkedList with
IntrusiveList in the Kernel Process class.
2021-06-07 09:42:55 +02:00
Ali Mohammad Pur
51c2c69357 AK+Everywhere: Disallow constructing Functions from incompatible types
Previously, AK::Function would accept _any_ callable type, and try to
call it when called, first with the given set of arguments, then with
zero arguments, and if all of those failed, it would simply not call the
function and **return a value-constructed Out type**.
This lead to many, many, many hard to debug situations when someone
forgot a `const` in their lambda argument types, and many cases of
people taking zero arguments in their lambdas to ignore them.
This commit reworks the Function interface to not include any such
surprising behaviour, if your function instance is not callable with
the declared argument set of the Function, it can simply not be
assigned to that Function instance, end of story.
2021-06-06 00:27:30 +04:30
Timothy Flynn
f8f36effc9 LibSQL: Limit the allowed depth of an expression tree
According to the definition at https://sqlite.org/lang_expr.html, SQL
expressions could be infinitely deep. For practicality, SQLite enforces
a maxiumum expression tree depth of 1000. Apply the same limit in
LibSQL to avoid stack overflow in the expression parser.

Fixes https://crbug.com/oss-fuzz/34859.
2021-06-05 23:48:18 +04:30
Sahan Fernando
d02e7b3811 LibWasm: Move Wasm::BytecodeInterpreter into its own header 2021-06-05 14:31:54 +04:30
Max Wipfli
99d5555134 AK: Do not trim away non-ASCII bytes when parsing URL
Because non-ASCII code points have negative byte values, trimming away
control characters requires checking for negative bytes values.

This also adds a test case with a URL containing non-ASCII code points.
2021-06-05 10:53:31 +02:00
Ali Mohammad Pur
be62e4d1d7 LibWasm: Load and instantiate tables
This commit is a fairly large refactor, mainly because it unified the
two different ways that existed to represent references.
Now Reference values are also a kind of value.
It also implements a printer for values/references instead of copying
the implementation everywhere.
2021-06-04 16:07:42 +04:30
R Smith
5a6f0ef1bc AK: Don’t drop lines between \r and \n in StringView::lines() (#7662)
StringView::lines() supports line-separators “\n”, “\r”, and “\r\n”.
The method will drop an entire line if it is surrounded by “\r”
and “\n” separators on the left and right sides respectively.
2021-06-04 12:06:08 +04:30
DexesTTP
e01f1c949f AK: Do not VERIFY on invalid code point bytes in UTF8View
The previous behavior was to always VERIFY that the UTF-8 bytes were
valid when iterating over the code points of an UTF8View. This change
makes it so we instead output the 0xFFFD 'REPLACEMENT CHARACTER'
code point when encountering invalid bytes, and keep iterating the
view after skipping one byte.

Leaving the decision to the consumer would break symmetry with the
UTF32View API, which would in turn require heavy refactoring and/or
code duplication in generic code such as the one found in
Gfx::Painter and the Shell.

To make it easier for the consumers to detect the original bytes, we
provide a new method on the iterator that returns a Span over the
data that has been decoded. This method is immediately used in the
TextNode::compute_text_for_rendering method, which previously did
this in a ad-hoc waay.

This also add tests for the new behavior in TestUtf8.cpp, as well
as reinforcements to the existing tests to check if the underlying
bytes match up with their expected values.
2021-06-03 18:28:27 +04:30
Max Wipfli
1c9d87c455 Tests: Add tests for most functions in AK/CharacterType.h 2021-06-03 13:31:46 +02:00
Brian Gianforcaro
ce74fce0df Tests: Add test coverage for AK::IntrusiveList reverse iterator support 2021-06-03 13:27:40 +02:00
Timothy Flynn
a870eac0eb LibSQL: Report a syntax error for unsupported LIMIT clause syntax
Rather than aborting when a LIMIT clause of the form 'LIMIT expr, expr'
is encountered, fail the parser with a syntax error. This will be nicer
for the user and fixes the following fuzzer bug:
https://crbug.com/oss-fuzz/34837
2021-06-03 08:30:13 +02:00
Ali Mohammad Pur
b15a5d6ada LibWasm: Ensure that value signs are preserved when casting
Also makes normal arithmetic operations more spec-compliant by actually
ignoring overflow on them.
2021-06-02 16:09:16 +04:30
Ali Mohammad Pur
56bf80251c LibWasm: Implement reference instructions (ref.{null,func,is_null}) 2021-06-02 16:09:16 +04:30
Timothy Flynn
ab79599a5e LibSQL: Return an error for empty common table expression lists
SQL::CommonTableExpressionList is required to be non-empty. Return an
error if zero common table expressions were parsed.

Fixes #7627
2021-06-01 23:48:21 +04:30
Ali Mohammad Pur
944855ca18 AK+Everywhere: Fix compiletime format parsing of replacement fields 2021-06-01 23:12:17 +04:30
Andreas Kling
c0d1a75881 AK: Strip leading/trailing C0-control-or-space in URLs correctly
We have to stop scanning once we hit a non-strippable character.
Add some tests to cover this.
2021-06-01 13:22:04 +02:00
Max Wipfli
f3fda59abd AK: Enable direct comparsion of Optional<T> and T
This patch introduces a new operator== to compare an Optional to its
contained type directly. If the Optional does not contain a value, the
comparison will always return false.

This also adds a test case for the new behavior as well as comparison
between Optional objects themselves.
2021-06-01 11:38:17 +02:00
Andreas Kling
407d6cd9e4 AK: Rename Utf8CodepointIterator => Utf8CodePointIterator 2021-06-01 09:45:52 +02:00
Max Wipfli
c7857f3572 Tests: Add more tests for AK::URL
This adds more tests for AK::URL. Furthermore, this also changes some
tests to conform to what the reworked URL class does (and the URL
specification mostly expects).
2021-06-01 09:28:05 +02:00
Max Wipfli
14506e8f5e AK: Implement Utf8CodepointIterator::peek(size_t)
This adds a peek method for Utf8CodepointIterator, which enables it to
be used in some parsing cases where peeking is necessary.

peek(0) is equivalent to operator*, expect that peek() does not contain
any assertions and will just return an empty Optional<u32>.

This also implements a test case for iterating UTF-8.
2021-06-01 09:28:05 +02:00
Andrew Kaster
0af192ff8d AK: Handle LEB128 encoded values that are too large for the result type
Previously, we would go crazy and shift things way out of bounds.
Add tests to verify that the decoding algorithm is safe around the
limits of the result type.
2021-05-31 14:25:27 +04:30
Andrew Kaster
c59cf0da07 AK: Add tests for LEB128 decoder 2021-05-31 14:25:27 +04:30
Andrew Kaster
7b4dc590e7 AK+Userland: Use akaster@serenityos.org for my copyright headers 2021-05-30 14:35:34 +01:00
Andreas Kling
c584421592 AK: Make HashTable::operator=(HashTable&&) clear the moved-from table
This is consistent with how other AK containers behave when moved from.
2021-05-30 14:34:32 +02:00
Ali Mohammad Pur
f387da4a90 LibTest+test-js: Add back the lost test262 parser test option
Fixes #7566.
2021-05-30 10:34:44 +01:00
Ali Mohammad Pur
90de1ded55 Kernel: Ensure that an unveil node with no permission is never accepted
Otherwise nodes inheriting from root may still be accessed with
`access(..., F_OK)`.
Also adds a test case to TestKernelUnveil about this behaviour.
2021-05-29 22:05:34 +02:00
Tim Schumacher
e87a9a7f9f Tests: Add tests for LexicalPath dirname handling 2021-05-27 18:21:36 +04:30
Andrew Kaster
a223ef3c4f Tests: Use ByteBuffer::create_zeroed in TestDeflate instead of memset
The round trip compress test wants the first half of the byte buffer to
be filled with random data, and the second half to be all zeroes. The
strategy of using memset on ByteBuffer::offset_pointer confuses
__builtin_memset_chk when building with -fsanitize=undefined. It thinks
that the buffer is using inline capacity when we can prove to ourselves
pretty easily that it's not. To avoid this, just create the buffer
zeroed to start, and then fill the first half with the random data.
2021-05-27 15:18:03 +02:00
Jesse Buhagiar
2b123cc592 AK: Implement AK::Stack 2021-05-26 16:36:53 +04:30
Gunnar Beutner
c81b3e1ee3 LibC: Implement strerror_r()
This implements the XSI-compliant version of strerror_r() - as opposed
to the GNU-specific variant.

The function explicitly saves errno so as to not accidentally change it
with one of the calls to other functions.
2021-05-25 17:36:02 +02:00