| Age | Commit message (Collapse) | Author |
|---|
|
std::unordered_map is faster than std::map in our use case,
reducing cache generation time by about 10% in my benchmark.
|
|
This converts all callers that read machine-generated data,
callers that might work with user input are not converted.
|
|
This is like isspace(), but ignores the current locale.
|
|
There is not much point and this is more readable.
Gbp-Dch: ignore
|
|
This is mostly a documentation issue, as the size we want to
read is always less than or equal to the size of the buffer,
so the return value will be the same as the size argument.
Nonetheless, people wondered about it, and it seems clearer
to just always use the return value.
|
|
This further improves our performance, and rred on uncompressed
files now spents 78% of its time in writing. Which means that
we should really look at buffering those.
|
|
The code uses memmove() to move parts of the buffer to the
front when the buffer is only partially read. By simply
reading one page at a time, the maximum size of bytes that
must be moved has a hard limit, and performance improves:
In one test case, consisting of a 430 MB Contents file,
and a 75K PDiff, applying the PDiff previously took about
48 seconds and now completes in 2 seconds.
Further speed up can be achieved by buffering writes, they
account for about 60% of the run-time now.
|
|
Gbp-Dch: ignore
|
|
And as we are at it lets fix the 'style' issue I introduced with the
filefd changes as well.
Reported-By: gcc -fsanitize's & cppcheck
Git-Dch: Ignore
|
|
We don't need the buffer that often - only for ReadLine - as it is only
occasionally used, so it is actually more efficient to allocate it if
needed instead of statically by default. It also allows the caller to
influence the buffer size instead of hardcoding it.
Git-Dch: Ignore
|
|
The default implementation of ReadLine was very naive by just reading
each character one-by-one. That is kinda okay for libraries implementing
compression as they have internal buffers (but still not great), but
while working with files directly or via a pipe as there is no buffer
there so all those reads are in fact system calls.
This commit introduces an internal buffer in the FileFd implementation
which is only used by ReadLine. The more low-level Read and all other
actions remain unbuffered – they just changed to deal with potential
"left-overs" in the buffer correctly.
Closes: 808579
|
|
If we use the library to compress xz, still try to understand and pick
up the arguments we would have used to call xz to figure out which level
the user wants us to use instead of defaulting to level 6 (which is the
default level of xz).
|
|
dpkg switched from CRC32 to CRC64 in
777915108d9d36d022dc4fc4151a615fc95e5032 with the message:
| This is the default CRC used by the xz command-line tool, align with
| it and switch from CRC32 to CRC64. It should provide slightly better
| detection against damaged data, at a negligible speed difference.
|
|
This isn't implementing any new features, it is "just" moving code
around from FileFd methods which decided on each call how to handle the
request by including all logic for all possible compressor backends in
the method body to a model in which backend-specifics are implemented in
a FileFdPrivate subclass. This avoids a big chunk of #ifdef's and should
make it a tiny bit more obvious which backend uses which code.
The execution of the idea is slightly uglified by the need to preserve
ABI and API which causes liberal befriending.
Git-Dch: Ignore
|
|
There's no point trying to read 0 bytes, so let's just not
do this and switch to a while loop like in Write().
Gbp-Dch: ignore
|
|
Turn the do-while loop into while loops, so it simply does nothing
if the Size is already 0.
This reverts commit c0b271edc2f6d9e5dea5ac82fbc911f0e3adfa7a which
introduced a fix for a specific instance of the issue in the
CopyFile() function.
Closes: #808381
|
|
On EOF, ToRead will be 0, which might trigger on some systems (e.g.
on the Hurd) an error due to the invalid byte count passed to write().
The whole loop already checks for ToRead != 0, so perform the writing
step only when there was actual data read.
Closes: #808381
|
|
Commit e977b8b9234ac5db32f2f0ad7e183139b988340d tries to make BufSize
calculated based on the size of the buffer; the problem is that
std::unique_ptr::size() returns a pointer to the data, so sizeof()
equals to the size of a pointer (later divided by sizeof(char), which
is 1). The result is that the CopyFile copies in chunks of 8 bytes,
which is not exactly ideal...
As solution, declare BufSize in advance, and use its value to allocate
the Buf array.
Closes: #808381
|
|
This helps writing test cases. Also adapt the test case that
expected 64-bit.
Nothing changes performance wise, the distribution of the hash
values remains intact.
|
|
This makes the test suite work on 32 bit-long platforms.
Gbp-Dch: ignore
|
|
Trying to clean up directories which do not exist seems rather silly if
you think about it, so let apt think about it and stop it.
Depends a bit on the caller if this is fixing anything for them as they
might try to acquire a lock or doing other clever things as apt does.
Closes: 807477
|
|
Regression of 1e064088bf7b3e29cd36d30760fb3e4143a1a49a (1.1~exp4) which
moved code around and renamed methods heavily ending up calling the
wrong method matching packagenames only instead of calling the full
array. Most commands work with versions, so this managed to fly under
the radar for quite a while.
Closes: 807870
|
|
If we can't work with the hashes we parsed from the Release file we
display now an error message if the Release file includes only weak
hashes instead of downloading the indexes and failing to verify them
with "Hash Sum mismatch" even through the hashes didn't mismatch (they
were just weak).
If for some (unlikely) reason we have got weak hashes only for
individual targets we will show a warning to this effect (again, befor
downloading and failing the index itself).
Closes: 806459
|
|
Reversing the parsing order ensures that we parse weaker hashes (like
SHA1) before we touch newer/stronger hashes (like SHA256) as the weaker
ones will usually be there for a longer time already with data already
present, which we would discard if we start with the strong one first.
The discarding is visible in the debug logs:
File X wasn't in the list for the first parsed hash! (history)
File X wasn't in the list for the first parsed hash! (patches)
which if file X is part of the patch-path means apt will not find a path and
fallback to acquire the whole file instead needlessly.
If file X isn't part of the patch-path that is no problem, so that
effects only the update-call which updates with patches coming from
before and after the addition of a new hash.
|
|
Reported-By: Manuel A. Fernandez Montecelo <mafm@debian.org>
Git-Dch: Ignore
|
|
Fixes a warning reported by gcc.
Gbp-Dch: ignore
|
|
With the package names now normalized to lower case, the caches
of affected systems need to be rebuild. Adjust the minor version
to trigger such a rebuild.
Gbp-Dch: ignore
|
|
dpkg does that when reading package files, so we should do
the same. This only deals with parsing names from binary
package paragraphs, it does not look at source package names
and/or the list of binaries in a dsc file.
Closes: #807012
|
|
required and important were swapped, leading to wrong
output.
Closes: #807523
Thanks: Manuel A. Fernandez Montecelo for discovering this
|
|
Gbp-Dch: ignore
|
|
We need to pass 0llu instead of 0 as the init value, otherwise
std::accumulate will calculate with ints.
Reported-by: Raphaël Hertzog
|
|
This should make it more obvious that CHANGEPATH is a placeholder which
apt will replace with a package specific path rather than a string
constant.
Mail-Reference: <87d1upgvaf.fsf@deep-thought.43-1.org>
Mail-Archive: https://lists.debian.org/debian-dak/2015/12/msg00005.html
|
|
Otherwise a user is subject to unexpected content-injection depending on
which directory she happens to start apt in. This also cleans up the code
requiring less implementation details in build-dep which is always good.
Technically, this is an ABI break as we override virtual methods, but
that they weren't overridden was a mistake resulting in pure classes,
which shouldn't be pure, so they were unusable – and as they are new in
1.1 nobody is using them yet (and hopefully ever as they are borderline
implementation details).
Closes: 806693
|
|
Regression of 14341a7ee1ca3dbcdcdbe10ad19b947ce23d972d.
Reported-By: Julian Andres Klode <jak@debian.org>
|
|
Git-Dch: Ignore
|
|
The implementation changed in
495b7a615a2d8f485beadf88c6ed298f5bbe50c2
Git-Dch: Ignore
Reported-By: Julian Andres Klode <jak@debian.org>
|
|
This reverts commit 7ac9386cb6e272625490fcf3e8183b45e28bbc43.
|
|
This reverts commit 28f24d3dad1844af316337d565ba2ebc11c8ce97.
This fails on Ubuntu as they build with -Bsymbolic-functions.
|
|
Dropping privileges is an involved process for code and system alike so
ideally we want to verify that all the work wasn't in vain. Stuff
designed to sidestep the usual privilege checks like fakeroot (and its
many alternatives) have their problem with this through, partly through
missing wrapping (#806521), partly as e.g. regaining root from an
unprivileged user is in their design. This commit therefore disables
most of these checks by default so that apt runs fine again in a
fakeroot environment.
Closes: 806475
|
|
This also deals with the unlikely case of groups being mentioned
multiple times or if the effective group isn't mentioned at all.
In practice, it is a debugging aid through like for #806475.
Git-Dch: Ignore
|
|
Git-Dch: Ignore
|
|
Seems like a simpler workaround than forcing a lower optimization
level just for this for all of apt.
See also:
https://bugs.launchpad.net/ubuntu/+source/gcc-5/+bug/1473674
|
|
Git-Dch: Ignore
|
|
Closes: #806422
|
|
If it does not exist disabled priv dropping as there is nothing
we can drop to. This will unblock people with special chroots
or systems that deleted the "_apt" user.
Closes: #806406
|
|
This fixes a regression caussed by commit
95278287f4e1eeaf5d96749d6fc9bfc53fb400d0
that moved the error detection of RunFds() later into the loop.
However this broke detecting issues like dead acquire methods.
Instead of relying on the global error state (which is bad)
we now pass a boolean value back from RunFds() and break on
false.
Closes: #806406
|
|
The manpages were fixed by Justin B Rye, lets deal with the rest now.
Git-Dch: Ignore
|
|
Git-Dch: Ignore
|
|
Reference mail:
https://lists.debian.org/debian-l10n-english/2015/11/msg00006.html
|
|
In 8d041b4f we made apt figure out based on the last Release file it has
if it should request a file or not given that the hashes changed or not.
So if we have a last Release file and do a request, do not sent a
Last-Modified header as we expect a change so much that a non-change
would indeed be an error. The Last-Modified header is therefore at best
ignored by the server, so sending it is just wasted effort. In the worst
case as time is a fragile thing the server decides against sending us an
update with the idea that we already have the latest content, which we
know for a fact that we haven't. Given that we sent less information to
the server our request is on its own also less identifiable as coming
from a returning or new user.
The disadvantage is that if we end up getting an old index file after
getting a new Release file from another mirror the old mirror will not
be able to tell us 'Hit', but instead sends us the complete file we
discard, but both lets us end up with the same error class in the end,
so the difference isn't big in practice.
|
