| Age | Commit message (Collapse) | Author |
|---|
|
The liblzma-based write code needs the same tweaks that the read code
already has to cope with the situation where lzma_code returns zero the
first time through because avail_out is zero, but will do more work if
called again.
This ports the read tweaks to the write code as closely as possible
(including matching comments etc.).
Closes: #751688
|
|
If we just reopened the file, we also need to reset the current
seek position when we reset the buffer, otherwise the code will
not try to seek to the position given to Skip (from 0), but will
try to seek to old offset + the position given to skip.
Closes: #812994, #813000
|
|
When writing into the buffer write to free() bytes starting
at getend(), instead of buffersize_max bytes at get()
-> get() is a read pointer.
This makes no difference in practice though, as we reset
the buffer before the call, so start = end = 0.
Gbp-Dch: ignore
|
|
We cannot just return false without setting an error,
as InternalWrite does not set one itself.
|
|
Microoptimization, but still gives a measurable 2-3% improvement
when using commands with lots of output like `apt list`.
|
|
It makes no sense to split a large block into multiple small
blocks, so when we have the chance to write them unbuffered,
do so.
|
|
We do not need the loop, FileFd::Private() handles this for us.
Gbp-Dch: ignore
|
|
We want to check whether the amount of free space is smaller
than the requested write size. Checking maxsize - size() is
incorrect for bufferstart >= 0, as size() = end - start.
Gbp-Dch: ignore
|
|
|
|
gcc correctly reports that we check for the same value twice, expect
that the manpage of read(2) tells us to do it for portability, so to
make both sides happy lets add a little #if'ing here.
Reported-By: gcc-6
Git-Dch: Ignore
|
|
APT::StringView is supposed to be a temporary measure, until support
for the standardized string_view is widely available. Introducing
additional unstandardized features just makes porting to the
standard version harder.
The constexpr constructor also won't have any real effect on most
systems, as the compiler will happily optimise the strlen() call
away for constant strings.
Gbp-Dch: ignore
|
|
The commit also adds a few trivial tests
Git-Dch: Ignore
|
|
The position returned is supposed to be the position of the character
counted from the start of the string, but if we used the substr calling
overloads the skipped over prefix wasn't considered. The pos parameter
of rfind had also the wrong semantic.
|
|
By storing the size of the string in the cache, we can make use of
it when comparing the names in the hashtable in pkgCache::FindGrp.
|
|
Hide the std::string overload instead of providing a
const char * one, the old variant was stupid.
Gbp-Dch: ignore
|
|
Use the same path for both comparisons, as the operator== path
is faster than just calling compare() - it avoids any comparison
if the size differs.
Gbp-Dch: ignore
|
|
Gbp-Dch: ignore
|
|
Gbp-Dch: ignore
|
|
Thanks: Niels Thykier for reporting on IRC
Gbp-Dch: ignore
|
|
This improves performance of the cache generation on my
ARM platform (4x Cortex A15) by about 10% to 20% from
2.35-2.50 to 2.1 seconds.
|
|
The class APT::StringView implements a drop-in replacement
for a subset of C++17 std::string_view() features. It will
be dropped at a later point and may not be used in public
interfaces.
|
|
This is a multiple of the page size and thus results in less
page faults, speeding up copying.
Also, while we're at at, unify all uses of that size in a
constant variable APT_BUFFER_SIZE.
|
|
Implement native support for LZ4 compression, using the official
lz4 library.
|
|
This makes code easier to read, and somewhat more correct.
Gbp-Dch: ignore
|
|
Gbp-Dch: ignore
|
|
This shuts up gcc
Gbp-Dch: ignore
|
|
To preserve compatibility, the new inline functions have _inline
as a suffix, and a macro defines the old names to refer to the
inline variants.
The old functions are still preserved for binary compatibility.
Also simplify the implementation of both functions.
|
|
Previously, if flush errored inside the loop, data could have
already been written to the wrapped descriptor without having
been removed from the buffer.
Also try to work around EINTR here. A better solution might be
to have the individual privates detect an interrupt and return
0 in such a case, instead of relying on errno being untouched
in between the syscall and the return from InternalWrite.
|
|
This avoids some issues with InternalWrite returning 0 because
it just cannot write stuff at the moment.
|
|
This is somewhat experimental right now, and might not work
for everyone, so it is on an opt-in basis.
|
|
The flush function can be used for buffered writers.
|
|
We will soon implement a buffered writing decorator and we will
need to forward attribute changes to those.
|
|
These can be used to implement write buffering
Gbp-Dch: ignore
|
|
Suggested by David.
Gbp-Dch: ignore
|
|
Gbp-Dch: ignore
|
|
We directly check if we are a hex digit in HexDigit, so use that
information.
[jak@debian.org: Commit message wording]
|
|
This converts all callers that read machine-generated data,
callers that might work with user input are not converted.
|
|
This is like isspace(), but ignores the current locale.
|
|
There is not much point and this is more readable.
Gbp-Dch: ignore
|
|
This is mostly a documentation issue, as the size we want to
read is always less than or equal to the size of the buffer,
so the return value will be the same as the size argument.
Nonetheless, people wondered about it, and it seems clearer
to just always use the return value.
|
|
This further improves our performance, and rred on uncompressed
files now spents 78% of its time in writing. Which means that
we should really look at buffering those.
|
|
The code uses memmove() to move parts of the buffer to the
front when the buffer is only partially read. By simply
reading one page at a time, the maximum size of bytes that
must be moved has a hard limit, and performance improves:
In one test case, consisting of a 430 MB Contents file,
and a 75K PDiff, applying the PDiff previously took about
48 seconds and now completes in 2 seconds.
Further speed up can be achieved by buffering writes, they
account for about 60% of the run-time now.
|
|
Gbp-Dch: ignore
|
|
And as we are at it lets fix the 'style' issue I introduced with the
filefd changes as well.
Reported-By: gcc -fsanitize's & cppcheck
Git-Dch: Ignore
|
|
We don't need the buffer that often - only for ReadLine - as it is only
occasionally used, so it is actually more efficient to allocate it if
needed instead of statically by default. It also allows the caller to
influence the buffer size instead of hardcoding it.
Git-Dch: Ignore
|
|
The default implementation of ReadLine was very naive by just reading
each character one-by-one. That is kinda okay for libraries implementing
compression as they have internal buffers (but still not great), but
while working with files directly or via a pipe as there is no buffer
there so all those reads are in fact system calls.
This commit introduces an internal buffer in the FileFd implementation
which is only used by ReadLine. The more low-level Read and all other
actions remain unbuffered – they just changed to deal with potential
"left-overs" in the buffer correctly.
Closes: 808579
|
|
If we use the library to compress xz, still try to understand and pick
up the arguments we would have used to call xz to figure out which level
the user wants us to use instead of defaulting to level 6 (which is the
default level of xz).
|
|
dpkg switched from CRC32 to CRC64 in
777915108d9d36d022dc4fc4151a615fc95e5032 with the message:
| This is the default CRC used by the xz command-line tool, align with
| it and switch from CRC32 to CRC64. It should provide slightly better
| detection against damaged data, at a negligible speed difference.
|
|
This isn't implementing any new features, it is "just" moving code
around from FileFd methods which decided on each call how to handle the
request by including all logic for all possible compressor backends in
the method body to a model in which backend-specifics are implemented in
a FileFdPrivate subclass. This avoids a big chunk of #ifdef's and should
make it a tiny bit more obvious which backend uses which code.
The execution of the idea is slightly uglified by the need to preserve
ABI and API which causes liberal befriending.
Git-Dch: Ignore
|
|
There's no point trying to read 0 bytes, so let's just not
do this and switch to a while loop like in Write().
Gbp-Dch: ignore
|
