| Age | Commit message (Collapse) | Author |
|---|
|
zstd is a compression algorithm developed by facebook. At level 19,
it is about 6% worse in size than xz -6, but decompression is multiple
times faster, saving about 40% install time, especially with eatmydata
on cloud instances.
|
|
Commit 89c4c588b275 ("fix from David Kalnischkies for the InRelease gpg
verification code (LP: #784473)") amended verification of cleartext
signatures by a check whether the file to be verified actually starts
with "-----BEGIN PGP SIGNATURE-----\n".
However cleartext signed InRelease files have been found in the wild
which use \r\n as line ending for this armor header line, presumably
generated by a Windows PGP client. Such files are incorrectly deemed
unsigned and result in the following (misleading) error:
Clearsigned file isn't valid, got 'NOSPLIT' (does the network require authentication?)
RFC 4880 specifies in 6.2 Forming ASCII Armor:
That is to say, there is always a line ending preceding the
starting five dashes, and following the ending five dashes. The
header lines, therefore, MUST start at the beginning of a line, and
MUST NOT have text other than whitespace following them on the same
line.
RFC 4880 does not seem to specify whether LF or CRLF is used as line
ending for armor headers, but CR is generally considered whitespace
(e.g. "man perlrecharclass"), hence using CRLF is legal even under
the assumption that LF must be used.
SplitClearSignedFile() is stripping whitespace (including CR) on lineend
already before matching the string, so StartsWithGPGClearTextSignature() is
adapted to use the same ignoring. As the earlier method is responsible
for what apt will end up actually parsing nowadays as signed/unsigned this
change has no implications for security.
Thanks: Lukas Wunner for detailed report & initial patch!
References: 89c4c588b275d098af33f36eeddea6fd75068342
Closes: 884922
|
|
LookupTag is a little helper to deal with rfc822-style strings we use in
apt e.g. to pass acquire messages around for cases in which our usual
rfc822 parser is too heavy. All the fields it had to deal with so far
were single line, but if they aren't it should really produce the right
output and not just return the first line. Error messages are a prime
candidate for becoming multiline as at the moment they are stripped of
potential newlines due to the previous insufficiency of LookupTag.
|
|
The casts are useless, but the reports show some where we can actually
improve the code by replacing them with better alternatives like
converting whatever int type into a string instead of casting to a
specific one which might in the future be too small.
Reported-By: gcc -Wuseless-cast
|
|
This caused a build failure in the test suite.
|
|
We have support for an netrc-like auth.conf file since 0.7.25 (closing
518473), but it was never documented in apt that it even exists and
netrc seems to have fallen out of usage as a manpage for it no longer
exists making the feature even more arcane.
On top of that the code was a bit of a mess (as it is written in c-style)
and as a result the matching of machine tokens to URIs also a bit
strange by checking for less specific matches (= without path) first.
We now do a single pass over the stanzas.
In practice early adopters of the undocumented implementation will not
really notice the differences and the 'new' behaviour is simpler to
document and more usual for an apt user.
Closes: #811181
|
|
Using different ways of opening files means we have different behaviour
and error messages for them, so by the same for all we can have more
uniformity for users and apt developers alike.
|
|
This makes it easier to see which headers includes what.
The changes were done by running
git grep -l '#\s*include' \
| grep -E '.(cc|h)$' \
| xargs sed -i -E 's/(^\s*)#(\s*)include/\1#\2 include/'
To modify all include lines by adding a space, and then running
./git-clang-format.sh.
|
|
Gbp-Dch: Ignore
|
|
In the intended usecase where this serves as a hack there is no problem
with double/single quotes being present as we write it to a log file
only, but nowadays our calling of apt-key produces a temporary config
file containing this "setting" as well and suddently quoting is
important as the config file syntax is allergic to it.
So the fix is to ignore all quoting whatsoever in the input and just
quote (with singles) the option values with spaces. That gives us 99% of
the time the correct result and the 1% where the quote is an integral
element of the option … doesn't exist – or has bigger problems than a
log file not containing the quote. Same goes for newlines in values.
LP: #1672710
|
|
This will avoid people from thinking that they have to do nothing
when they change the set of files.
Gbp-Dch: ignore
|
|
Our implementation of wildcards was rudimentary. It worked for some
common ones, but it was also broken: For example, armel matched any-armel,
but should match any-arm.
With this commit, we load the correct tables from dpkg. Supported are
both triplets and quadruplet tables (the latter introduced in dpkg 1.18.11).
There are some odd things we have to deal with in the cache filter for
historical and API reasons:
* The character "*" must be accepted as an alternative to any - in fact
it may appear anywhere in the wildcard as we also allow fnmatch() style
wildcard matching on the commandline.
* The code might get passed an arch with a minus at the end, for example
the cmdline "install apt:any-arm-" will first try to check if any-arm-
is a valid architecture. We deal with this by rejecting any wildcard
ending in a minus.
* Triplets are actually implemented by extending them to faux quadruplets
- by prepending a "base" component for the architecture tuple, and "any"
if there is a wildcard component.
Once we have constructed a wildcard, it is transformed into an fnmatch()
expression for historical reasons. In the future, we should really get a
tuple class and implement matching in a better, more explicit way.
This does for now though - it passes all the test cases and accepts all
things it should accept.
Closes: #748936
Thanks: James Clarke <jrtc27@jrtc27.com> for the initial patch
|
|
Run the test for kfreebsd-i386 and amd64 and pass "amd64" as
an additional argument to the function. This tests that the
argument is used and thus ParseDepends returns the amd64
results even on a different architecture like i386.
|
|
Clearsigned files like InRelease, .dsc, .changes and co can potentially
include unsigned or additional messages blocks ignored by gpg in
verification, but a potential source of trouble in our own parsing
attempts – and an unneeded risk as the usecases for the clearsigned
files we deal with do not reasonably include unsigned parts (like emails
or some such).
This commit changes the silent ignoring to warnings for now to get an
impression on how widespread unintended unsigned parts are, but
eventually we want to turn these into hard errors.
|
|
The test test-handle-redirect-as-used-mirror-change serves multiple
clients at the same time, so the order of the output is undefined and
once in a while the two clients will intermix their lines causing the
grep we perform on it later to fail making our tests fail.
Solved by introducing client-specific logfiles which we all grep and
sort the result to have the results more stable.
Git-Dch: Ignore
|
|
|
|
This fixes a regression introduced in
commit 8f858d560e3b7b475c623c4e242d1edce246025a
don't leak FD in AutoProxyDetect command return parsing
which accidentally made the proxy autodetection code also read
the scripts output on stderr, not only on stdout when it switched
the code from popen() to Popen().
Reported-By: Tim Small <tim@seoss.co.uk>
|
|
If a non-existing source directory is specified, try finding
the system gtest library. Debian derived distributions are
a bit strange because they only ship the source code and
not the library...
|
|
Gbp-Dch: Ignore
Reported-By: gcc -Wmissing-declarations
|
|
Git-Dch: Ignore
|
|
We use clock() as a very cheap way of getting a "random" value, but the
manpage warns that this could return -1, so we should be dealing with
this. Additionally, e.g. on hurd-i386 the value increases only slowly –
to slow for our fast running tests for randomness hence producing the
same range in both samples, so we introduce a simple busy-wait loop (as
clock is counting processor time used by the program) in the test which
delays the second sample just enough making our randomness a bit more
predictable.
|
|
Comparing floating numbers is always fun and in this instance a 9 < 9.0
is "somehow" true on hurd-i386 letting the tests fail by reporting that
too much progress achieved. A bit mysterious, but with some rework we
can use code which avoids dealing with the floats in this way entirely
and make our testcases happy.
|
|
If the URI had no password the username was ignored
|
|
Bye, bye, old friend.
|
|
Add support for our GTest based unit tests. By default, CMake will
look in /usr/src/gtest for the external GTest project, but this can
be overriden by defining GTEST_ROOT when invoking cmake.
Gbp-Dch: ignore
|
|
Very unlikely, but if the parent is /dev/null, the child empty and the
grandchild a value we returned /dev/null/value which doesn't exist, so
hardly a problem, but for best operability we should be consistent in
our work and return /dev/null always.
|
|
We deploy atomic renames for some files, but these renames also happen
if something about the file failed which isn't really the point of the
exercise…
Closes: 828908
|
|
As reported upstream in
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71556
the implementation of std::get_time is currently not as accepting as
strptime is, especially in how hours should be formatted.
Just reverting 9febc2b238e1e322dce1f94ecbed46d595893b52 would be
possible, but then we would reopen the problems fixed by it, so instead
I opted here for a rewrite of the parsing logic which makes this method
a lot longer, but at least it provides the same benefits as the rewrite
in std::get_time was intended to give us and decouples us from the fix
of the issue in the standard library implementation of GCC.
LP: 1593583
|
|
Seen first in #826783, but as this buglog also shows leaked uncompressed
files as well we don't close it just yet.
|
|
HTTP/1.1 hardcodes GMT (RFC 7231 §7.1.1.1) and what is good enough for the
internet must be good enough for us™ as we reuse the implementation
internally to parse (most) dates we encounter in various places like the
Release files with their Date and Valid-Until header fields.
Implementing a fully timezone aware parser just feels too hard for no
effective benefit as it would take 5+ years (= until LTS's are out of
fashion) until a repository could use non-UTC dates and expect it to
work. Not counting non-apt implementations which might or might not
only want to encounter UTC here as well.
As a bonus, this eliminates the use of an instance of setlocale in
libapt.
Closes: 819697
|
|
libapt allows to configure compressors to be used by its system via
configuration implemented in 03bef78461c6f443187b60799402624326843396,
but that was never really documented and also only partly working, which
also explains why the tests weren't using it…
|
|
|
|
On launchpad #1558484 a user reports that @ in the authentication tokens
parsing of sources.list isn't working in an older (precise) version. It
isn't the recommended way of specifying passwords and co (auth.conf is),
but we can at least test for regressions (and in this case test at all…
who was that "clever" boy disabling a test with exit……… oh, nevermind.
Git-Dch: Ignore
|
|
SHA1 is not reasonably secure anymore, so we should not consider it
usable anymore. The test suite is adjusted to account for this.
|
|
This tests the fix for #812994, #813000
Gbp-Dch: ignore
|
|
APT has a different understanding than dpkg (#748936) what matches and
what doesn't match an architecture specification as it isn't converting
back (and forward) to Debian triplets. That has to eventually be solved
some way or the other, but until that happens we change the matching in
apt so that porters can continue their work on non-gnu libc-ports even
if policy doesn't specify that yet (and dpkg just supporting it "by
accident" via triplets).
The initial patch was reformatted, fixed in terms of patterns containing
"any-any", dealing with expanding an arch without libc to gnu while a
pattern expands libc to any, the parsedepends test was fixed (the new
if's were inserted one step too early) and another test just for the
specifications added.
Closes: #812212
Thanks: Bálint Réczey for initial patch
|
|
Avoid the dependency on a specific current path for the tar test and
ensure that _system is correctly initialized (gcc-6 runs into a segfault
otherwise and with it fixed starts to depend on the multi-arch
configuration of the running system… not good).
Git-Dch: Ignore
|
|
APT::StringView is supposed to be a temporary measure, until support
for the standardized string_view is widely available. Introducing
additional unstandardized features just makes porting to the
standard version harder.
The constexpr constructor also won't have any real effect on most
systems, as the compiler will happily optimise the strlen() call
away for constant strings.
Gbp-Dch: ignore
|
|
The commit also adds a few trivial tests
Git-Dch: Ignore
|
|
APT usually deals with perfectly formatted files generated automatically
be other programs – and as it has to parse multiple MBs of such files it
tries to be fast rather than forgiving.
This was always a problem if we reused this parser for files with a
deb822 syntax which are mostly written by hand however, like
apt_preferences or the deb822-style sources as these can include stray
newlines and more importantly comments all over the place.
As a stopgap we had pkgUserTagSection which deals at least with comments
before and after a given stanza, but comments in between weren't really
supported and now that we support parsing debian/control for e.g.
build-dep we face the full comment problem e.g. with comments inbetween
multi-line fields (like Build-Depends).
We can't easily deal with this on the pkgTagSection level as the interface
gives access to 'raw' char-pointers for performance reasons so we would
need to optionally add a buffer here on which we could remove comments
to hand out pointers into this buffer instead. The interface is quite
large already and supports writing stanzas as well, which does not
support comments at all either. So while in future it might make sense
to have a parser setup which deals with and keeps comments in this
commit we opt for the simpler solution for now: We officially declare
that pkgTagSection does not support comments and instead expect the
caller to deal with them, which in our case is pkgTagFile:
pkgTagFile is extended with an additional mode which can deal with
comments by dropping them from the buffer which will later form the
input of pkgTagSection. The actual implementation is slightly more
complex than this sentence suggests at first on one hand to have good
performance and on the other to allow jumping directly to stanzas with
offsets collected in a previous run (like our cache generation does it
for example).
|
|
Regression introduced in 8710a36a01c0cb1648926792c2ad05185535558e,
but such fields are unlikely in practice as it is just as simple to not
have a field at all with the same result of not having a value.
Closes: 808102
|
|
Testing /usr as TMPDIR assumes that GetTempDir() cannot use it
because it cannot write to it; this is true for non-root users, but
not so much for root.
Since root can access everything, perform this particular test case
only when not running as root.
Closes: #808383
|
|
required and important were swapped, leading to wrong
output.
Closes: #807523
Thanks: Manuel A. Fernandez Montecelo for discovering this
|
|
Seems like a simpler workaround than forcing a lower optimization
level just for this for all of apt.
See also:
https://bugs.launchpad.net/ubuntu/+source/gcc-5/+bug/1473674
|
|
space-gapping: '-o option= value'
That is a very old feature (straight from 1998), but it is super
surprising if you try setting empty values and instead get error
messages or a non-empty value as the next parameter is treated as the
value – which could have been empty, so if for some reason you need a
compatible way of setting an empty value try: '-o option="" ""'.
I can only guess that the idea was to support '-o option value', but we
survived 17 years without it, we will do fine in the future I guess.
Similar is the case for '-t= testing' even through '-t testing' existed
before and the code even tried to detect mistakes like '-t= -b' … all
gone now.
Technically that is as its removing a feature replacing it with another
a major interface break. In practice I really hope for my and their
sanity that nobody was using this; but if for some reaon you do: Remove
the space and be done.
I found the patch and the bugreport actually only after the fact, but
its reassuring that others are puzzled by this as well and hence a
thanks is in perfect order here as the patch is practical identical
[expect that this one here adds tests and other bonus items].
Thanks: Daniel Hartwig for initial patch.
Closes: 693092
|
|
In 0940230d we started dropping privileges for file (and a bit later for
copy, too) with the intend of uniforming this for all methods. The
commit message says that the source will likely fail based on the
compressors already – and there isn't much secret in the repository
content. After all, after apt has run the update everyone can access the
content via apt anyway…
There are sources through which worked before which are mostly
single-deb (and those with the uncompressed files available).
The first one being especially surprising for users maybe, so instead of
failing, we make it so that apt detects that it can't access a source as
_apt and if so doesn't drop (for all sources!) privileges – but we limit
this to file/copy, so the uncompress which might be needed will still
fail – but that failed before this regression.
We display a notice about this, mostly so that if it still fails (e.g.
compressed) the user has some idea what is wrong.
Closes: 805069
|
|
Limit the field length to a char to avoid bogus FF for
utf-8 characters with the default length.
Closes: 799123
|
|
Reported-By: cppcheck
Git-Dch: Ignore
|
|
The wrapping will fail in the best case and actually end up deleting
/dev/null in the worst case. Given that there is no point in trying to
write atomically to /dev/null as you can't read from it again just
ignore these flags if higher level code ends up trying to use them on
/dev/null.
Git-Dch: Ignore
|
|
Based on a discussion with Niels Thykier who asked for Contents-all this
implements apt trying for all architecture dependent files to get a file
for the architecture all, which is treated internally now as an official
architecture which is always around (like native). This way arch:all
data can be shared instead of duplicated for each architecture requiring
the user to download the same information again and again.
There is one problem however: In Debian there is already a binary-all/
Packages file, but the binary-any files still include arch:all packages,
so that downloading this file now would be a waste of time, bandwidth
and diskspace. We therefore need a way to decide if it makes sense to
download the all file for Packages in Debian or not. The obvious answer
would be a special flag in the Release file indicating this, which would
need to default to 'no' and every reasonable repository would override
it to 'yes' in a few years time, but the flag would be there "forever".
Looking closer at a Release file we see the field "Architectures", which
doesn't include 'all' at the moment. With the idea outlined above that
'all' is a "proper" architecture now, we interpret this field as being
authoritative in declaring which architectures are supported by this
repository. If it says 'all', apt will try to get all, if not it will be
skipped. This gives us another interesting feature: If I configure a
source to download armel and mips, but it declares it supports only
armel apt will now print a notice saying as much. Previously this was a
very cryptic failure. If on the other hand the repository supports mips,
too, but for some reason doesn't ship mips packages at the moment, this
'missing' file is silently ignored (= that is the same as the repository
including an empty file).
The Architectures field isn't mandatory through, so if it isn't there,
we assume that every architecture is supported by this repository, which
skips the arch:all if not listed in the release file.
|
