Commit graph

575 commits

Author SHA1 Message Date
Andreas Kling
ec6bceaa08 Kernel: Support thread-local storage
This patch adds support for TLS according to the x86 System V ABI.
Each thread gets a thread-specific memory region, and the GS segment
register always points _to a pointer_ to the thread-specific memory.

In other words, to access thread-local variables, userspace programs
start by dereferencing the pointer at [gs:0].

The Process keeps a master copy of the TLS segment that new threads
should use, and when a new thread is created, they get a copy of it.
It's basically whatever the PT_TLS program header in the ELF says.
2019-09-07 15:55:36 +02:00
Andreas Kling
bcfdf9ffa7 AK: Add a useful align_up_to(value, power_of_two) function 2019-09-07 15:39:26 +02:00
Andreas Kling
d10ca2b010 AK: When printf assert on unsupported specifier, specify which one!
We were asserting without saying why. That's a bit unhelpful. :^)
2019-09-06 20:27:58 +02:00
Andreas Kling
73fdbba59c AK: Rename <AK/AKString.h> to <AK/String.h>
This was a workaround to be able to build on case-insensitive file
systems where it might get confused about <string.h> vs <String.h>.

Let's just not support building that way, so String.h can have an
objectively nicer name. :^)
2019-09-06 15:36:54 +02:00
Andreas Kling
fb39e46d3d Utf8View: Try fixing the travis-ci build
There's some overload ambiguity when doing Utf8View("literal")
2019-09-05 19:06:39 +02:00
Sergey Bugaev
55197ed4ef AK: Log UTF-8 validation errors 2019-09-05 16:37:39 +02:00
Sergey Bugaev
c379f43d2a AK: Add some more utility methods to Utf8View 2019-09-05 16:37:39 +02:00
Andreas Kling
cc98ea1956 Json: Add serializer fast-path for string values
Passing these through the generic JsonValue path was causing us to
instantiate temporary JsonValues that incurred a heap allocation.
This avoids that by adding specialized overloads for string types.
2019-09-04 14:40:11 +02:00
Conrad Pankoff
87808e535a AK: Fix printf %x padding and %p length 2019-09-03 15:23:13 +02:00
Conrad Pankoff
40daefd3dc AK: Abort on unknown printf formatting characters
Right now if we encounter an unknown character, printf (and its related
functions) fail in a really bad way, where they forget to pull things off
the stack. This usually leads to a crash somewhere else, which is hard to
debug.

This patch makes printf abort as soon as it encounters a formatting
character that it can't handle. This is not the optimal solution, but it
is an improvement for debugging.
2019-09-02 08:29:21 +02:00
Conrad Pankoff
a92939b766 AK: Support %i as an alias for %d in printf 2019-09-02 08:28:31 +02:00
Conrad Pankoff
4a4e66b2d0 Kernel/AK: Add is_zero helpers for IP and MAC addresses 2019-08-29 06:25:06 +02:00
Sergey Bugaev
5d3696174b AK: Add a Utf8View type for iterating over UTF-8 codepoints
Utf8View wraps a StringView and implements begin() and end() that
return a Utf8CodepointIterator, which parses UTF-8-encoded Unicode
codepoints and returns them as 32-bit integers.

This is the first step towards supporting emojis in Serenity ^)
https://github.com/SerenityOS/serenity/issues/490
2019-08-28 13:46:02 +02:00
Conrad Pankoff
970e0147f7 AK: Make printf %x actually work properly
When printing hex numbers, we were printing the wrong thing sometimes. This
was because we were dividing the digit to print by 15 instead of 16. Also,
dividing by 16 is the same as shifting four bits to the right, which is a
bit closer to our actual intention in this case, so let's use a shift
instead.
2019-08-28 06:43:04 +02:00
Sergey Bugaev
216b7b3b80 JSON: Port JsonArray and JsonObject serialization to serializers
This way, primitive JsonValue serialization is still handled by
JsonValue::serialize(), but JsonArray and JsonObject serialization
always goes through serializer classes. This is no less efficient
if you have the whole JSON in memory already.
2019-08-27 14:56:09 +02:00
Sergey Bugaev
56f5c14d86 JSON: Add JSON serializers
These are two new types that allow serializing JSON on-the-fly
as it's generated, without building the whole JSON in memory
first.
2019-08-27 14:56:09 +02:00
Andreas Kling
e29fd3cd20 Kernel: Display virtual addresses as V%p instead of L%x
The L was a leftover from when these were called linear addresses.
2019-08-26 11:31:58 +02:00
Andreas Kling
8f50d75184 AK: Make HashTable.h compile inside the SDL2 port 2019-08-25 17:47:48 +02:00
Andreas Kling
d38bd3935b AK: Add StringView::hash()
This grabs the hash from the underlying StringImpl if there is one,
otherwise it's computed on the fly.
2019-08-25 06:45:31 +02:00
Andreas Kling
bb32fd8bfa AK: Add HashMap::find() with customizable finder callback
This will allow clients to search the map without having to instantiate
a key value.
2019-08-25 06:45:27 +02:00
Andreas Kling
cd8278e489 AK: Add String::operator==(StringView)
Comparing a String to a StringView would instantiate a temporary
String just for the comparison. Let's not do that. :^)
2019-08-25 06:45:19 +02:00
Andreas Kling
a00419ed77 AK: Optional::operator bool() should consume the Optional
We use consumable annotations to catch bugs where you get the .value()
of an Optional before verifying that it's okay.

The bug here was that only has_value() would set the consumed state,
even though operator bool() does the same job.
2019-08-25 06:45:09 +02:00
Andreas Kling
56eaf9b033 AK: Make FileSystemPath better at handling relative paths
Relative paths now canonicalize into a string starting with "./"
Previously, "foo" would be canonicalized as "/foo" which was clearly
not right.
2019-08-23 19:58:16 +02:00
Andreas Kling
a5cdd0afa5 NonnullPtrVector: Add ptr_at() getters for accessing the NonnullPtr
Normally you want to access the T&, but sometimes you need to grab at
the NonnullPtr, for example when moving it out of the Vector before
a call to remove(). This is generally a weird pattern, but I don't have
a better solution at the moment.
2019-08-19 19:04:52 +02:00
Andreas Kling
5465795dc3 AK: The printf family was mixing up case and alternate form settings 2019-08-18 16:14:21 +02:00
Andreas Kling
e3f3c980bf IntrusiveList: Make Iterator::operator* return a T&
This makes iteration a little more pleasant :^)
2019-08-17 11:25:32 +02:00
Andreas Kling
5122caf9a8 LogStream: Prefix userspace dbg() output with "ProcessName(PID): "
Using the new get_process_name() syscall, we can automatically prefix
all userspace debug logging.

Hopefully this is more helpful than annoying. We'll find out! :^)
2019-08-15 20:55:45 +02:00
Andreas Kling
d64e698bf1 AK: Add a simple TriState type
enum class TriState : u8 { False, True, Unknown };
2019-08-15 20:53:12 +02:00
Andreas Kling
2349dc1a21 StringView: Add StringView::operator==(StringView)
Previously we'd implicitly convert the second StringView to a String
when comparing two StringViews, which is obviously not what we wanted.
2019-08-15 14:09:27 +02:00
Andreas Kling
eaa9cf58f5 AK: Use int_hash() to generate less idiotic hashes for {Nonnull,}OwnPtr 2019-08-14 21:21:15 +02:00
Andreas Kling
1ac963b5c8 JsonParser: "" is an empty string, not a null value 2019-08-14 15:01:42 +02:00
Andreas Kling
fdcff7d15e AK: Make it possible to use HashMap<K, NonnullOwnPtr>::get()
Add the concept of a PeekType to Traits<T>. This is the type we'll
return (wrapped in an Optional) from HashMap::get().

The PeekType for OwnPtr<T> and NonnullOwnPtr<T> is const T*,
which means that HashMap::get() will return an Optional<const T*> for
maps-of-those.
2019-08-14 11:47:38 +02:00
Andreas Kling
f75b1127b2 OwnPtr: Add a way to turn an OwnPtr into a NonnullOwnPtr
Okay, so, OwnPtr<T>::release_nonnull() returns a NonnullOwnPtr<T>.
It assumes that the OwnPtr is non-null to begin with.

Note that this removes the value from the OwnPtr, as there can only be
a single owner.
2019-08-14 11:04:19 +02:00
Conrad Pankoff
06743932b8 AK: Support width/alt/caps/padding modifiers for %x in printf 2019-08-13 18:15:58 +02:00
Andreas Kling
2c7b0b8893 Vector: Use memcpy to implement remove() for trivial types
This is a lot faster than the generic code path.
Also added some unit testing for this.
2019-08-12 11:07:31 +02:00
Andreas Kling
fb636389d6 URL: Add some convenience constructors 2019-08-10 19:31:37 +02:00
Andreas Kling
ed43770b2f AK: Add a basic URL class to help us handle URL's
We're gonna need these as we start to write more networking programs.
2019-08-10 17:30:35 +02:00
Sergey Bugaev
79f867238a printf: Support dynamic fill widths
The printf formatting mini-language actually allows you
to pass a '*' character in place of the fill width specification,
in which case it eats one of the passed in arguments and uses it
as width, so implement that.
2019-08-10 08:46:22 +02:00
Andreas Kling
865a1b913c AK: Add Optional<T>(const U&)
This replaces Optional<T>(U&&) which clang-tidy complained may hide the
regular copy and move constructors. That's a good point, clang-tidy,
and I appreciate you pointing that out!
2019-08-08 18:34:59 +02:00
Andreas Kling
bb9909548b AK: Add an iterator class for InlineLinkedList
This makes it possible to iterate over these with range-for. :^)
2019-08-08 13:39:40 +02:00
Andreas Kling
9889d170b9 JsonValue: Add as_string_or(String)
Return the contained string if the value *is* a string, otherwise it
returns the alternative string passed in the parameter.
2019-08-07 22:03:25 +02:00
Andreas Kling
f6998b1817 JSON: Templatize the JSON serialization code
This makes it possible to use something other than a StringBuilder for
serialization (and to produce something other than a String.) :^)
2019-08-07 21:29:32 +02:00
Andreas Kling
43ec733b61 AK: Add a basic unit test for FileSystemPath
Just to make sure that things are on the up-and-up.
2019-08-07 21:07:02 +02:00
Andreas Kling
2e416b1b87 Vector: Add a test for growing a Vector beyond its inline capacity 2019-08-07 20:43:13 +02:00
Andreas Kling
5096eaa845 AK: Add a FixedArray<T> container
This is a simple array wrapper that knows its size. It has begin/end
so you can use range-for. It also has a resize() that reallocates.
2019-08-07 20:05:36 +02:00
Andreas Kling
5487f81b5d Vector: Use memcpy when dynamically growing Vectors of trivial types 2019-08-07 15:35:23 +02:00
Andreas Kling
6da6ca64d2 Vector: Use TypedTransfer in more parts of Vector
Make more Vector-of-trivial-type operations go fast :^)
2019-08-07 15:25:34 +02:00
Andreas Kling
e8e85f5457 Vector: Use memcmp for comparing two vectors with trivial elements 2019-08-07 15:05:10 +02:00
Andreas Kling
6d97caf124 JsonParser: Scan ahead to find the first special char in quoted strings
This allows us to take advantage of the now-optimized (to do memmove())
Vector::append(const T*, int count) for collecting these strings.

This is a ~15% speedup on the load_4chan_catalog benchmark.
2019-08-07 11:57:51 +02:00
Andreas Kling
b48b6c0caa Vector: Use memmove() for moving trivial types around more
This can definitely be improved with better trivial type detection and
by using the TypedTransfer template in more places.

It's a bit annoying that we can't get <type_traits> in Vector.h since
it's included in the toolchain compilation before we have libstdc++.
2019-08-07 11:55:20 +02:00
Andreas Kling
60c25228ee AK: Fix -Wconsumed warnings in Optional move-ctor and move-assign
Our protocol says we have to call has_value() before release_value().
The code was already safe, but the compiler had no way of knowing that.
2019-08-07 07:17:52 +02:00
Andreas Kling
2abfab2892 ELFLoader: Remove an uninteresting debug log message
This is showing up at the boundary between kernel and userspace stack
frames in backtraces, and looks silly.
2019-08-06 20:49:36 +02:00
Conrad Pankoff
73c998dbfc Kernel: Refactor TCP/IP stack
This has several significant changes to the networking stack.

* Significant refactoring of the TCP state machine. Right now it's
  probably more fragile than it used to be, but handles quite a lot
  more of the handshake process.
* `TCPSocket` holds a `NetworkAdapter*`, assigned during `connect()` or
  `bind()`, whichever comes first.
* `listen()` is now virtual in `Socket` and intended to be implemented
  in its child classes
* `listen()` no longer works without `bind()` - this is a bit of a
  regression, but listening sockets didn't work at all before, so it's
  not possible to observe the regression.
* A file is exposed at `/proc/net_tcp`, which is a JSON document listing
  the current TCP sockets with a bit of metadata.
* There's an `ETHERNET_VERY_DEBUG` flag for dumping packet's content out
  to `kprintf`. It is, indeed, _very debug_.
2019-08-06 16:21:17 +02:00
Andreas Kling
151e6a1818 AK: Fix leak in Optional(Optional&&)
We were *copying* the other Optional's value and then marking it as not
having a value. This prevented us from ever destroying the original.
2019-08-05 22:27:47 +02:00
Andreas Kling
9553ecfe01 AK: Optional::operator=(Optional&&) should clear movee's has_value bit
We were forgetting to clear m_has_value in the Optional being moved
from when using operator=(Optional&&).
2019-08-05 21:47:36 +02:00
Sergey Bugaev
55d7810fab AK: Do not keep an open FD in MmappedFile
We only need an open FD while establishing the mapping,
and can close it immediately after mmap() call.
2019-08-05 16:04:31 +02:00
Andreas Kling
0d0230ab10 JsonParser: Fold extract_while() into parse_number()
It wasn't unsed anywhere else anyway, and this is actually ~1% faster
on the load_4chan_catalog benchmark.
2019-08-04 20:23:46 +02:00
Andreas Kling
a846ee76ea AK: Add VectorIterator::index()
A simple helper function that extracts the index of an iterator.
Note that the index is not valid if the iterator is end().
2019-08-04 20:18:18 +02:00
Andreas Kling
2a89bb2ac4 Vector: Add find() and some iterator improvements
Vector now has find() just like HashTable. I also made the iterator
comparison functions const-correct.
2019-08-04 19:21:08 +02:00
Andreas Kling
539985f4fe HashTable: Use the Bucket type in some places over SinglyLinkedList<T>
This is just for consistency, and we might want to switch to another
bucket type some day.
2019-08-04 19:20:20 +02:00
Andreas Kling
72b69b82bb JsonParser: Oops, fix build. 2019-08-04 18:59:06 +02:00
Andreas Kling
4e004a664f JsonParser: Cache the last seen string starting with each possible char
Keep a 256-entry string cache during parse to avoid creating some new
strings when possible. This cache is far from perfect but very cheap.
Since none of the strings are transient, this only costs us a couple of
pointers and a bit of ref-count manipulation.

The cache hit rate on 4chan_catalog.json is ~33% and the speedup on
the load_4chan_catalog benchmark is ~7%.
2019-08-04 18:41:24 +02:00
Andreas Kling
3eb1a7f8f8 AK: Add a benchmark for parsing 4chan catalog JSON
I was able to get parsing time down to about 1/3 of the original time
by using callgrind+kcachegrind. There's definitely more improvements
that can be made here, but I'm gonna be happy with this for now. :^)
2019-08-04 11:57:32 +02:00
Andreas Kling
b62a12c687 JsonParser: Some minor optimizations
- Return more specific types from parse_array() and parse_object().
- Don't create a throwaway String in extract_while().
- Use a StringView in parse_number() to avoid a throwaway String.
2019-08-04 11:47:21 +02:00
Andreas Kling
93596dc00d JsonObject: Add move constructors for JsonObject(Json{Array,Object}&&)
This also helps avoid JsonValue copying during parse.
2019-08-04 11:46:31 +02:00
Andreas Kling
82826104e0 Json: Add efficient copy and move constructors for Json{Array,Object}
This helps avoid copying JsonValues during parsing.
2019-08-04 11:45:16 +02:00
Andreas Kling
cce2ea9bb0 AK: Add StringView::to_int()
This is a shameless copy-paste of String::to_int(). We should find some
way to share this code between String and StringView instead of having
two duplicate copies like this.
2019-08-04 11:44:20 +02:00
Andreas Kling
c55129e573 JsonParser: Use Vector<char, 1024> instead of StringBuilder in parsing
This is a 10-12% speedup on the 4chan thread catalog JSON.
2019-08-04 10:38:15 +02:00
Andreas Kling
210550d4b3 JsonArray: Expose the value vector to the outside for convenience
Sometimes it's easier to just work with a const Vector<JsonValue>&,
so give clients the option of doing that.
2019-08-04 10:05:02 +02:00
Andreas Kling
6797f71e73 JsonParser: When encountering \uXXXX, just emit a "?" for now. 2019-08-04 08:58:45 +02:00
Andreas Kling
18aeda2e0d BufferStream: It's not an error to read up to the very end of a buffer
Don't signal an error before we've reached the very end of the buffer.
2019-08-03 19:20:05 +02:00
Andreas Kling
6e956f2a0b BufferStream: Add operator<<(const char*) overload
Ext2FS was relying on the presence of a (StringView) overload here and
when I added a (bool) overload, it started choosing that for literals.
2019-08-03 19:02:48 +02:00
Andreas Kling
b04dae7faa BufferStream: Support "bool" as a streaming type 2019-08-03 17:02:42 +02:00
Andreas Kling
c5903ec4bb AK: Remove two redundant RefPtr constructors.
We already have constructors for "const T*" and "const T&" so we don't
need to have non-const variants.
2019-08-03 14:16:05 +02:00
Andreas Kling
fe25426ee4 AK: Simplify RefPtr and NonnullRefPtr's leak_ref() functions
Use AK::exchange() to switch out the internal storage. Also mark these
functions with [[nodiscard]] to provoke an compile-time error if they
are called without using the return value.
2019-08-02 12:05:09 +02:00
Andreas Kling
15866714da AK: Add anti-null assertions in RefPtr.
This gives us better error messages when dereferencing null RefPtrs.
2019-08-02 12:00:43 +02:00
Andreas Kling
cbc1272810 AK: Fix ref leaks in RefPtr assignment operators.
Many of the RefPtr assignment operators would cause ref leaks when we
call them to assign a pointer that's already the one kept.
2019-08-02 11:56:55 +02:00
Andreas Kling
06f82901b7 AK: Explicitly delete NonnullRefPtr::operator=(RefPtr).
This gives us much better error messages when you try to use them.
Without this change, it would complain about the absence of functions
named ref() and deref() on RefPtr itself. With it, we instead get a
"hey, this function is deleted" error.

Change operator=(T&) to operator=T(const T&) also, to keep assigning
a const T& to a NonnullRefPtr working.
2019-08-02 11:51:28 +02:00
Andreas Kling
6db879ee66 AK: Fix ref leak in NonnullRefPtr::operator=(T&).
We would leak a ref when assigning a T& to a NonnullRefPtr that already
contains that same T.
2019-08-02 11:35:05 +02:00
Andreas Kling
d9cc3e453c AK: Add assertions when dereferencing an OwnPtr.
This will make it immediately obvious what the problem is when you're
dereferencing a null OwnPtr.
2019-08-02 10:34:40 +02:00
Andreas Kling
a9a1a5dfa9 AK: Add a test for iterating a HashTable during clear (should assert)
Ideally we should also verify that the assertion actually happens,
but we need some support in the TestSuite framework for that.
2019-08-02 09:25:35 +02:00
Andreas Kling
6560116b67 TestSuite: Hijack the ASSERT macros during unit tests.
Instead of aborting the program when we hit an assertion, just print a
message and keep going.

This allows us to write tests that provoke assertions on purpose.
2019-08-02 09:23:03 +02:00
Andreas Kling
31793b8f3a AK: Fix typo in the WeakPtr test. Behavior was actually correct.
Also remove an unused variable.
2019-08-02 09:21:42 +02:00
Andreas Kling
4e59300650 AK: Fix typo in TestVector.cpp, oops. 2019-08-02 08:59:26 +02:00
Andreas Kling
9230b42f28 AK: Use Vector::empend() a bit in the unit tests, and fix a bug.
There was a bug in the "prepend_vector_object" test but it was masked
by us not printing failures. (The bug was that we were adding three
elements to the "objects" vector and then checking that another
vector called "more_objects" indeed had three elements. Oops!)
2019-08-01 16:24:31 +02:00
Andreas Kling
8434548f14 TestSuite: Actually print failed comparions.. :^) 2019-08-01 16:22:50 +02:00
Andreas Kling
eeff0cd570 AK: Don't allow constructing an OwnPtr from a const NonnullOwnPtr&
OwnPtr's must move around, they can't be copy constructed.
2019-08-01 15:46:18 +02:00
Andreas Kling
79ce75d862 AK: Add Vector::empend().
This is a complement to append() that works by constructing the new
element in-place via placement new and forwarded constructor arguments.

The STL calls this emplace_back() which looks ugly, so I'm inventing
a nice word for it instead. :^)
2019-08-01 15:35:45 +02:00
Andreas Kling
a3ee35510f AK: Make Bitmap movable but not copyable.
We were falling back to an incorrect compiler-generated copy ctor for
this class, and let's not do that.

Found by PVS-Studio.
2019-08-01 11:34:36 +02:00
Andreas Kling
2923d39106 JsonParser: Merge the parsing of '\n' and '\r' in quoted strings 2019-08-01 10:45:37 +02:00
Andreas Kling
f10e850644 HashTable: Assert on iteration attempt over table during clear/rehash
It doesn't seem sane to try to iterate over a HashTable while it's in
the middle of being cleared. Since this might cause strange problems,
this patch adds an assertion if an iterator is constructed during
clear() or rehash() of a HashTable.
2019-07-31 10:23:24 +02:00
Robin Burchell
a82c60b22a Add Result<>, to use with/complement Error
An operation often has two pieces of underlying information:

* the data returned as a result from that operation
* an error that occurred while retrieving that data

Merely returning the data is not good enough. Result<> allows exposing
both the data, and the underlying error, and forces (via clang's
consumable attribute) you to check for the error before you try to
access the data.
2019-07-31 09:06:39 +02:00
Robin Burchell
7dd25141cd Add Error<>
Put simply, Error<> is a way of forcing error handling onto an API user.

Given a function like:

    bool might_work();

The following code might have been written previously:

    might_work(); // but what if it didn't?

The easy way to work around this is of course to [[nodiscard]] might_work.
But this doesn't work for more complex cases like, for instance, a
hypothetical read() function which might return one of _many_ errors
(typically signalled with an int, let's say).

    int might_read();

In such a case, the result is often _read_, but not properly handled. Like:

    return buffer.substr(0, might_read()); // but what if might_read returned an error?

This is where Error<> comes in:

    typedef Error<int, 0> ReadError;
    ReadError might_read();

    auto res = might_read();
    if (might_read.failed()) {
        switch (res.value()) {
        case EBADF:
            ...
        }
    }

Error<> uses clang's consumable attributes to force failed() to be
checked on an Error instance. If it's not checked, then you get smacked.
2019-07-31 09:06:39 +02:00
Robin Burchell
28362fcc57 Optional: Add consumable checks
These will, when building with clang, prevent using Optional::value
without first checking Optional::has_value() - at compile time.
2019-07-31 09:06:39 +02:00
rhin123
a175e76948 TextEditor: Include extension during SaveAs
When we save-as in the text editor we now auto-populate GFilePicker /w
the current name & extension.
2019-07-29 20:46:31 +02:00
Andreas Kling
57c29491a3 Kernel+AK: Remove AK/StdLibExtras.cpp, moving kernel stuff to Kernel/.
We had some kernel-specific gizmos in AK that should really just be in the
Kernel subdirectory instead. The only thing remaining after moving those
was mmx_memcpy() which I moved to the ARCH(i386)-specific section of
LibC/string.cpp.
2019-07-29 11:58:44 +02:00
Andreas Kling
66db6f4f92 AK: Add Queue::clear() to clear out a Queue. 2019-07-28 21:34:47 +02:00
Andreas Kling
6f397e23f1 ByteBuffer: Add slice_view(). Works like slice() but makes a wrapper only.
So we already have ByteBuffer::wrap() which is like a StringView for random
data. This might not be the best abstraction actually, but this will be
immediately useful so let's add it.
2019-07-27 15:26:51 +02:00
Andreas Kling
2f373a27a2 AK: Add RELEASE_ASSERT in non-Serenity builds. 2019-07-26 22:15:54 +02:00
Andreas Kling
fc9a1a1328 AK: Fix NonnullRefPtr<T>::operator=(NonnullRefPtr<U>). 2019-07-26 08:04:33 +02:00