| Age | Commit message (Collapse) | Author |
|---|
|
Add support for our GTest based unit tests. By default, CMake will
look in /usr/src/gtest for the external GTest project, but this can
be overriden by defining GTEST_ROOT when invoking cmake.
Gbp-Dch: ignore
|
|
Very unlikely, but if the parent is /dev/null, the child empty and the
grandchild a value we returned /dev/null/value which doesn't exist, so
hardly a problem, but for best operability we should be consistent in
our work and return /dev/null always.
|
|
We deploy atomic renames for some files, but these renames also happen
if something about the file failed which isn't really the point of the
exercise…
Closes: 828908
|
|
As reported upstream in
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71556
the implementation of std::get_time is currently not as accepting as
strptime is, especially in how hours should be formatted.
Just reverting 9febc2b238e1e322dce1f94ecbed46d595893b52 would be
possible, but then we would reopen the problems fixed by it, so instead
I opted here for a rewrite of the parsing logic which makes this method
a lot longer, but at least it provides the same benefits as the rewrite
in std::get_time was intended to give us and decouples us from the fix
of the issue in the standard library implementation of GCC.
LP: 1593583
|
|
Seen first in #826783, but as this buglog also shows leaked uncompressed
files as well we don't close it just yet.
|
|
HTTP/1.1 hardcodes GMT (RFC 7231 §7.1.1.1) and what is good enough for the
internet must be good enough for us™ as we reuse the implementation
internally to parse (most) dates we encounter in various places like the
Release files with their Date and Valid-Until header fields.
Implementing a fully timezone aware parser just feels too hard for no
effective benefit as it would take 5+ years (= until LTS's are out of
fashion) until a repository could use non-UTC dates and expect it to
work. Not counting non-apt implementations which might or might not
only want to encounter UTC here as well.
As a bonus, this eliminates the use of an instance of setlocale in
libapt.
Closes: 819697
|
|
libapt allows to configure compressors to be used by its system via
configuration implemented in 03bef78461c6f443187b60799402624326843396,
but that was never really documented and also only partly working, which
also explains why the tests weren't using it…
|
|
|
|
On launchpad #1558484 a user reports that @ in the authentication tokens
parsing of sources.list isn't working in an older (precise) version. It
isn't the recommended way of specifying passwords and co (auth.conf is),
but we can at least test for regressions (and in this case test at all…
who was that "clever" boy disabling a test with exit……… oh, nevermind.
Git-Dch: Ignore
|
|
SHA1 is not reasonably secure anymore, so we should not consider it
usable anymore. The test suite is adjusted to account for this.
|
|
This tests the fix for #812994, #813000
Gbp-Dch: ignore
|
|
APT has a different understanding than dpkg (#748936) what matches and
what doesn't match an architecture specification as it isn't converting
back (and forward) to Debian triplets. That has to eventually be solved
some way or the other, but until that happens we change the matching in
apt so that porters can continue their work on non-gnu libc-ports even
if policy doesn't specify that yet (and dpkg just supporting it "by
accident" via triplets).
The initial patch was reformatted, fixed in terms of patterns containing
"any-any", dealing with expanding an arch without libc to gnu while a
pattern expands libc to any, the parsedepends test was fixed (the new
if's were inserted one step too early) and another test just for the
specifications added.
Closes: #812212
Thanks: Bálint Réczey for initial patch
|
|
Avoid the dependency on a specific current path for the tar test and
ensure that _system is correctly initialized (gcc-6 runs into a segfault
otherwise and with it fixed starts to depend on the multi-arch
configuration of the running system… not good).
Git-Dch: Ignore
|
|
APT::StringView is supposed to be a temporary measure, until support
for the standardized string_view is widely available. Introducing
additional unstandardized features just makes porting to the
standard version harder.
The constexpr constructor also won't have any real effect on most
systems, as the compiler will happily optimise the strlen() call
away for constant strings.
Gbp-Dch: ignore
|
|
The commit also adds a few trivial tests
Git-Dch: Ignore
|
|
APT usually deals with perfectly formatted files generated automatically
be other programs – and as it has to parse multiple MBs of such files it
tries to be fast rather than forgiving.
This was always a problem if we reused this parser for files with a
deb822 syntax which are mostly written by hand however, like
apt_preferences or the deb822-style sources as these can include stray
newlines and more importantly comments all over the place.
As a stopgap we had pkgUserTagSection which deals at least with comments
before and after a given stanza, but comments in between weren't really
supported and now that we support parsing debian/control for e.g.
build-dep we face the full comment problem e.g. with comments inbetween
multi-line fields (like Build-Depends).
We can't easily deal with this on the pkgTagSection level as the interface
gives access to 'raw' char-pointers for performance reasons so we would
need to optionally add a buffer here on which we could remove comments
to hand out pointers into this buffer instead. The interface is quite
large already and supports writing stanzas as well, which does not
support comments at all either. So while in future it might make sense
to have a parser setup which deals with and keeps comments in this
commit we opt for the simpler solution for now: We officially declare
that pkgTagSection does not support comments and instead expect the
caller to deal with them, which in our case is pkgTagFile:
pkgTagFile is extended with an additional mode which can deal with
comments by dropping them from the buffer which will later form the
input of pkgTagSection. The actual implementation is slightly more
complex than this sentence suggests at first on one hand to have good
performance and on the other to allow jumping directly to stanzas with
offsets collected in a previous run (like our cache generation does it
for example).
|
|
Regression introduced in 8710a36a01c0cb1648926792c2ad05185535558e,
but such fields are unlikely in practice as it is just as simple to not
have a field at all with the same result of not having a value.
Closes: 808102
|
|
Testing /usr as TMPDIR assumes that GetTempDir() cannot use it
because it cannot write to it; this is true for non-root users, but
not so much for root.
Since root can access everything, perform this particular test case
only when not running as root.
Closes: #808383
|
|
required and important were swapped, leading to wrong
output.
Closes: #807523
Thanks: Manuel A. Fernandez Montecelo for discovering this
|
|
Seems like a simpler workaround than forcing a lower optimization
level just for this for all of apt.
See also:
https://bugs.launchpad.net/ubuntu/+source/gcc-5/+bug/1473674
|
|
space-gapping: '-o option= value'
That is a very old feature (straight from 1998), but it is super
surprising if you try setting empty values and instead get error
messages or a non-empty value as the next parameter is treated as the
value – which could have been empty, so if for some reason you need a
compatible way of setting an empty value try: '-o option="" ""'.
I can only guess that the idea was to support '-o option value', but we
survived 17 years without it, we will do fine in the future I guess.
Similar is the case for '-t= testing' even through '-t testing' existed
before and the code even tried to detect mistakes like '-t= -b' … all
gone now.
Technically that is as its removing a feature replacing it with another
a major interface break. In practice I really hope for my and their
sanity that nobody was using this; but if for some reaon you do: Remove
the space and be done.
I found the patch and the bugreport actually only after the fact, but
its reassuring that others are puzzled by this as well and hence a
thanks is in perfect order here as the patch is practical identical
[expect that this one here adds tests and other bonus items].
Thanks: Daniel Hartwig for initial patch.
Closes: 693092
|
|
In 0940230d we started dropping privileges for file (and a bit later for
copy, too) with the intend of uniforming this for all methods. The
commit message says that the source will likely fail based on the
compressors already – and there isn't much secret in the repository
content. After all, after apt has run the update everyone can access the
content via apt anyway…
There are sources through which worked before which are mostly
single-deb (and those with the uncompressed files available).
The first one being especially surprising for users maybe, so instead of
failing, we make it so that apt detects that it can't access a source as
_apt and if so doesn't drop (for all sources!) privileges – but we limit
this to file/copy, so the uncompress which might be needed will still
fail – but that failed before this regression.
We display a notice about this, mostly so that if it still fails (e.g.
compressed) the user has some idea what is wrong.
Closes: 805069
|
|
Limit the field length to a char to avoid bogus FF for
utf-8 characters with the default length.
Closes: 799123
|
|
Reported-By: cppcheck
Git-Dch: Ignore
|
|
The wrapping will fail in the best case and actually end up deleting
/dev/null in the worst case. Given that there is no point in trying to
write atomically to /dev/null as you can't read from it again just
ignore these flags if higher level code ends up trying to use them on
/dev/null.
Git-Dch: Ignore
|
|
Based on a discussion with Niels Thykier who asked for Contents-all this
implements apt trying for all architecture dependent files to get a file
for the architecture all, which is treated internally now as an official
architecture which is always around (like native). This way arch:all
data can be shared instead of duplicated for each architecture requiring
the user to download the same information again and again.
There is one problem however: In Debian there is already a binary-all/
Packages file, but the binary-any files still include arch:all packages,
so that downloading this file now would be a waste of time, bandwidth
and diskspace. We therefore need a way to decide if it makes sense to
download the all file for Packages in Debian or not. The obvious answer
would be a special flag in the Release file indicating this, which would
need to default to 'no' and every reasonable repository would override
it to 'yes' in a few years time, but the flag would be there "forever".
Looking closer at a Release file we see the field "Architectures", which
doesn't include 'all' at the moment. With the idea outlined above that
'all' is a "proper" architecture now, we interpret this field as being
authoritative in declaring which architectures are supported by this
repository. If it says 'all', apt will try to get all, if not it will be
skipped. This gives us another interesting feature: If I configure a
source to download armel and mips, but it declares it supports only
armel apt will now print a notice saying as much. Previously this was a
very cryptic failure. If on the other hand the repository supports mips,
too, but for some reason doesn't ship mips packages at the moment, this
'missing' file is silently ignored (= that is the same as the repository
including an empty file).
The Architectures field isn't mandatory through, so if it isn't there,
we assume that every architecture is supported by this repository, which
skips the arch:all if not listed in the release file.
|
|
The general idea is: A small paragraph on the tool itself as a
description, a list of the most used (!= all) commands available in the
tool, a remark where to find more information on the tool and its
commands (aka: in the manpage) and finally a common block referring to
even more manpages. In exchange options are completely omitted from the
output as well as deprecated or obscure commands. (Better) Information
about them is available in the manpages anyway and the few options which
were listed before were also the least interesting ones (-o -c -q and co
are hardly of interest for someone totally new looking to find info by
asking for help and anyone with a bit of experience doesn't need this
short list. Those would need a list of options applying to the command
they call, but they are too numerous and command specific to list them
sanely in this context.
|
|
Its not as simple as I initially thought to abstract this enough to make
it globally usable, so lets not pollute global namespace with this for
now.
Git-Dch: Ignore
|
|
This ensures that location strings loaded from a location specified via
configuration (Dir::Locale) effect the help messages for commands.
Git-Dch: Ignore
|
|
Especially with apt now, it can be useful to set an option only for apt
and not for apt-get. Using a binary-specific subtree which is merged into
the root seems like a simple enough trick to achieve this.
|
|
The previous implementation was still a bit unstable in terms of failing
at times. Lets try if we have more luck with this one.
Git-Dch: Ignore
|
|
Initializing a random number generator with the time since epoch could
be good enough, but reaches its limits in test code as the 100
iterations might very well happen in the same second and hence the seed
number is always the same… clock() has a way lower resolution so it
changes more often and not unimportant: If many users start the update
at the same time it isn't to unlikely the SRV record will be ordered in
the same second choosing the same for them all, but it seems less likely
that the exact same clock() time has passed for them.
And if I have to touch this, lets change a few other things as well to
make me and/or compilers a bit happier (clang complained about the usage
of a GNU extension in the testcase for example).
|
|
We want to declare some hashes as not enough for security, so that a
user will need --allow-unauthenticated or similar to get data secured
only by those hashes, but we can still us these hashes for integrity
checks if we got them.
|
|
Git-Dch: ignore
|
|
Also add "Debug::Acquire::SrvRecs" debug option and the option
"Acquire::EnableSrvRecods" to allow disabling this lookup.
|
|
C++11 adds the 'override' specifier to mark that a method is overriding
a base class method and error out if not. We hide it in the APT_OVERRIDE
macro to ensure that we keep compiling in pre-c++11 standards.
Reported-By: clang-modernize -add-override -override-macros
Git-Dch: Ignore
|
|
Having two different formats in the same file is very dirty and causes
external tools to fail hard trying to parse them. It is probably not a
good idea for them to parse them in the first place, but they do and we
shouldn't break them if there is a better way.
So we solve this issue for now by giving our deb822 format a new
filename extension ".sources" which unsupporting applications are likely
to ignore an can begin gradually moving forward rather than waiting for
the unknown applications to catch up.
Currently and for the forseeable future apt is going to support both
with the same feature set as documented in the manpage, with the
longtime plan of adopting the 'new' format as default, but that is a
long way to go and might get going more from having an easier time
setting options than from us pushing it explicitely.
|
|
Various small leaks here and there. Nothing particularily big, but still
good to fix. Found by the sanitizers while running our testcases.
Reported-By: gcc -fsanitize
Git-Dch: Ignore
|
|
More warnings are always better.
Git-Dch: Ignore
|
|
This fixes the tests on systems where usrmerge is installed.
Gbp-dch: ignore
|
|
To have a chance to keep the ABI for a while we need all three to team
up. One of them missing and we might loose, so ensuring that they are
available is a very tedious but needed task once in a while.
Git-Dch: Ignore
|
|
It is a rather strange sight that index items use SiteOnly which strips
the Path, while e.g. deb files are downloaded with NoUserPassword which
does not. Important to note here is that for the file transport Path is
pretty important as there is no Host which would be displayed by Site,
which always resulted in "interesting" unspecific errors for "file:".
Adding a 'middle' ground between the two which does show the Path but
potentially modifies it (it strips a pending / at the end if existing)
solves this "file:" issue, syncs the output and in the end helps to
identify which file is meant exactly in progress output and co as a
single site can have multiple repositories in different paths.
|
|
Having every item having its own code to verify the file(s) it handles
is an errorprune process and easy to break, especially if items move
through various stages (download, uncompress, patching, …). With a giant
rework we centralize (most of) the verification to have a better
enforcement rate and (hopefully) less chance for bugs, but it breaks the
ABI bigtime in exchange – and as we break it anyway, it is broken even
harder.
It shouldn't effect most frontends as they don't deal with the acquire
system at all or implement their own items, but some do and will need to
be patched (might be an opportunity to use apt on-board material).
The theory is simple: Items implement methods to decide if hashes need to
be checked (in this stage) and to return the expected hashes for this
item (in this stage). The verification itself is done in worker message
passing which has the benefit that a hashsum error is now a proper error
for the acquire system rather than a Done() which is later revised to a
Failed().
|
|
While it is mostly busywork to rewrite all instances it actually fixes
bugs as the data storage used by the new method is std::string rather
than a char*, the later mostly created by c_str() from a std::string
which the caller has to ensure keeps in scope – something apt-ftparchive
actually didn't ensure and relied on copy-on-write behavior instead
which c++11 forbids and hence the new default gcc abi doesn't use it.
|
|
TFRewrite is okay, but it has obscure limitations (256 Tags), even more
obscure bugs (order for renames is defined by the old name) and the
interface is very c-style encouraging bad usage like we do it in
apt-ftparchive passing massive amounts of c_str() from std::string in.
The old-style is marked as deprecated accordingly. The next commit will
fix all places in the apt code to not use the old-style anymore.
|
|
Git-Dch: Ignore
|
|
Methods get told which hashes are expected by the acquire system, which
means we can use this list to restrict what we calculate in the methods
as any extra we are calculating is wasted effort as we can't compare it
with anything anyway.
Adding support for a new hash algorithm is therefore 'free' now and if a
algorithm is no longer provided in a repository for a file, we
automatically stop calculating it.
In practice this results in a speed-up in Debian as we don't have SHA512
here (so far), so we practically stop calculating it.
|
|
We just need it for unit tests and our debian/rules file actually skips
calling them if nocheck is given… but this fails anyhow as we declared a
hard-dependency on it. Demoting the error to a warning in configuration
and adding a test in the 'make test' path with a friendly message allows
nocheck to be useful again.
(Running unit tests is fully encouraged of course, but bootstrappers and
co do not need to be burdened with this stuff)
|
|
This isn't testing much of the 'complex' parts,
but its better than nothing for now.
Git-Dch: Ignore
|
|
Working with strings c-style is complicated and error-prune,
so by converting to c++ style we gain some simplicity and
avoid buffer overflows by later extensions.
Git-Dch: Ignore
|
