| Age | Commit message (Collapse) | Author |
|---|
|
Based on a discussion with Niels Thykier who asked for Contents-all this
implements apt trying for all architecture dependent files to get a file
for the architecture all, which is treated internally now as an official
architecture which is always around (like native). This way arch:all
data can be shared instead of duplicated for each architecture requiring
the user to download the same information again and again.
There is one problem however: In Debian there is already a binary-all/
Packages file, but the binary-any files still include arch:all packages,
so that downloading this file now would be a waste of time, bandwidth
and diskspace. We therefore need a way to decide if it makes sense to
download the all file for Packages in Debian or not. The obvious answer
would be a special flag in the Release file indicating this, which would
need to default to 'no' and every reasonable repository would override
it to 'yes' in a few years time, but the flag would be there "forever".
Looking closer at a Release file we see the field "Architectures", which
doesn't include 'all' at the moment. With the idea outlined above that
'all' is a "proper" architecture now, we interpret this field as being
authoritative in declaring which architectures are supported by this
repository. If it says 'all', apt will try to get all, if not it will be
skipped. This gives us another interesting feature: If I configure a
source to download armel and mips, but it declares it supports only
armel apt will now print a notice saying as much. Previously this was a
very cryptic failure. If on the other hand the repository supports mips,
too, but for some reason doesn't ship mips packages at the moment, this
'missing' file is silently ignored (= that is the same as the repository
including an empty file).
The Architectures field isn't mandatory through, so if it isn't there,
we assume that every architecture is supported by this repository, which
skips the arch:all if not listed in the release file.
|
|
The general idea is: A small paragraph on the tool itself as a
description, a list of the most used (!= all) commands available in the
tool, a remark where to find more information on the tool and its
commands (aka: in the manpage) and finally a common block referring to
even more manpages. In exchange options are completely omitted from the
output as well as deprecated or obscure commands. (Better) Information
about them is available in the manpages anyway and the few options which
were listed before were also the least interesting ones (-o -c -q and co
are hardly of interest for someone totally new looking to find info by
asking for help and anyone with a bit of experience doesn't need this
short list. Those would need a list of options applying to the command
they call, but they are too numerous and command specific to list them
sanely in this context.
|
|
Its not as simple as I initially thought to abstract this enough to make
it globally usable, so lets not pollute global namespace with this for
now.
Git-Dch: Ignore
|
|
This ensures that location strings loaded from a location specified via
configuration (Dir::Locale) effect the help messages for commands.
Git-Dch: Ignore
|
|
Especially with apt now, it can be useful to set an option only for apt
and not for apt-get. Using a binary-specific subtree which is merged into
the root seems like a simple enough trick to achieve this.
|
|
The previous implementation was still a bit unstable in terms of failing
at times. Lets try if we have more luck with this one.
Git-Dch: Ignore
|
|
Initializing a random number generator with the time since epoch could
be good enough, but reaches its limits in test code as the 100
iterations might very well happen in the same second and hence the seed
number is always the same… clock() has a way lower resolution so it
changes more often and not unimportant: If many users start the update
at the same time it isn't to unlikely the SRV record will be ordered in
the same second choosing the same for them all, but it seems less likely
that the exact same clock() time has passed for them.
And if I have to touch this, lets change a few other things as well to
make me and/or compilers a bit happier (clang complained about the usage
of a GNU extension in the testcase for example).
|
|
We want to declare some hashes as not enough for security, so that a
user will need --allow-unauthenticated or similar to get data secured
only by those hashes, but we can still us these hashes for integrity
checks if we got them.
|
|
Git-Dch: ignore
|
|
Also add "Debug::Acquire::SrvRecs" debug option and the option
"Acquire::EnableSrvRecods" to allow disabling this lookup.
|
|
C++11 adds the 'override' specifier to mark that a method is overriding
a base class method and error out if not. We hide it in the APT_OVERRIDE
macro to ensure that we keep compiling in pre-c++11 standards.
Reported-By: clang-modernize -add-override -override-macros
Git-Dch: Ignore
|
|
Having two different formats in the same file is very dirty and causes
external tools to fail hard trying to parse them. It is probably not a
good idea for them to parse them in the first place, but they do and we
shouldn't break them if there is a better way.
So we solve this issue for now by giving our deb822 format a new
filename extension ".sources" which unsupporting applications are likely
to ignore an can begin gradually moving forward rather than waiting for
the unknown applications to catch up.
Currently and for the forseeable future apt is going to support both
with the same feature set as documented in the manpage, with the
longtime plan of adopting the 'new' format as default, but that is a
long way to go and might get going more from having an easier time
setting options than from us pushing it explicitely.
|
|
Various small leaks here and there. Nothing particularily big, but still
good to fix. Found by the sanitizers while running our testcases.
Reported-By: gcc -fsanitize
Git-Dch: Ignore
|
|
More warnings are always better.
Git-Dch: Ignore
|
|
This fixes the tests on systems where usrmerge is installed.
Gbp-dch: ignore
|
|
To have a chance to keep the ABI for a while we need all three to team
up. One of them missing and we might loose, so ensuring that they are
available is a very tedious but needed task once in a while.
Git-Dch: Ignore
|
|
It is a rather strange sight that index items use SiteOnly which strips
the Path, while e.g. deb files are downloaded with NoUserPassword which
does not. Important to note here is that for the file transport Path is
pretty important as there is no Host which would be displayed by Site,
which always resulted in "interesting" unspecific errors for "file:".
Adding a 'middle' ground between the two which does show the Path but
potentially modifies it (it strips a pending / at the end if existing)
solves this "file:" issue, syncs the output and in the end helps to
identify which file is meant exactly in progress output and co as a
single site can have multiple repositories in different paths.
|
|
Having every item having its own code to verify the file(s) it handles
is an errorprune process and easy to break, especially if items move
through various stages (download, uncompress, patching, …). With a giant
rework we centralize (most of) the verification to have a better
enforcement rate and (hopefully) less chance for bugs, but it breaks the
ABI bigtime in exchange – and as we break it anyway, it is broken even
harder.
It shouldn't effect most frontends as they don't deal with the acquire
system at all or implement their own items, but some do and will need to
be patched (might be an opportunity to use apt on-board material).
The theory is simple: Items implement methods to decide if hashes need to
be checked (in this stage) and to return the expected hashes for this
item (in this stage). The verification itself is done in worker message
passing which has the benefit that a hashsum error is now a proper error
for the acquire system rather than a Done() which is later revised to a
Failed().
|
|
While it is mostly busywork to rewrite all instances it actually fixes
bugs as the data storage used by the new method is std::string rather
than a char*, the later mostly created by c_str() from a std::string
which the caller has to ensure keeps in scope – something apt-ftparchive
actually didn't ensure and relied on copy-on-write behavior instead
which c++11 forbids and hence the new default gcc abi doesn't use it.
|
|
TFRewrite is okay, but it has obscure limitations (256 Tags), even more
obscure bugs (order for renames is defined by the old name) and the
interface is very c-style encouraging bad usage like we do it in
apt-ftparchive passing massive amounts of c_str() from std::string in.
The old-style is marked as deprecated accordingly. The next commit will
fix all places in the apt code to not use the old-style anymore.
|
|
Git-Dch: Ignore
|
|
Methods get told which hashes are expected by the acquire system, which
means we can use this list to restrict what we calculate in the methods
as any extra we are calculating is wasted effort as we can't compare it
with anything anyway.
Adding support for a new hash algorithm is therefore 'free' now and if a
algorithm is no longer provided in a repository for a file, we
automatically stop calculating it.
In practice this results in a speed-up in Debian as we don't have SHA512
here (so far), so we practically stop calculating it.
|
|
We just need it for unit tests and our debian/rules file actually skips
calling them if nocheck is given… but this fails anyhow as we declared a
hard-dependency on it. Demoting the error to a warning in configuration
and adding a test in the 'make test' path with a friendly message allows
nocheck to be useful again.
(Running unit tests is fully encouraged of course, but bootstrappers and
co do not need to be burdened with this stuff)
|
|
This isn't testing much of the 'complex' parts,
but its better than nothing for now.
Git-Dch: Ignore
|
|
Working with strings c-style is complicated and error-prune,
so by converting to c++ style we gain some simplicity and
avoid buffer overflows by later extensions.
Git-Dch: Ignore
|
|
Checking the return value of this (and many other calls) in this
testcase is a good idea, so we do it now.
Reported-By: cppcheck
Git-Dch: Ignore
|
|
"did you intend to multiply instead?" is what cppcheck helpful says and
it is absolutely right. Doesn't make a whole lot of a difference though
as we are talking about 'char' in this testcase, but just to be sure.
Reported-By: cppcheck
Git-Dch: Ignore
|
|
The testcases have far worse problems if these ever end up being NULL
and/or are not given a value by the method called, but clang is right to
warn about it, just that we don't want to fix it in testcases…
Git-Dch: Ignore
|
|
Central methods of our infrastructure like this one responsible for
communication with our methods shouldn't be more complicated then they
have to and not claim to have (albeit unlikely) bugs.
While I am not sure about having improved the first part, the bug is now
gone and a few explicit tests check that it stays that way, so nobody
will notice the difference (hopefully) – expect that this should a very
tiny bit faster as well as we don't manually proceed through the string.
Git-Dch: Ignore
|
|
It is a very simple hashstring, which is why it isn't contributing to
the usability of a list of them, but it is also trivial to check and
calculate, so it doesn't hurt checking it either as it can combined even
with the simplest other hashes greatly complicate attacks on them as you
suddenly need a same-size hash collision, which is usually a lot harder
to achieve.
|
|
Git-Dch: Ignore
|
|
Private temporary directories as created by e.g. libpam-tmpdir are nice,
but they are also very effective in preventing our priviledge dropping
to work as TMPDIR will be set to a directory only root has access to, so
working with it as _apt will fail. We circumvent this by extending our
check for a usable TMPDIR setting by checking access rights.
Closes: 765951
|
|
|
|
|
|
This used to work before we implemented a stricter commandline parser
and e.g. the dd-schroot-cmd command constructs commandlines like this.
Reported-By: Helmut Grohne
|
|
Conflicts:
apt-pkg/acquire-item.cc
apt-pkg/acquire-item.h
apt-pkg/cachefilter.h
configure.ac
debian/changelog
|
|
Git-Dch: Ignore
|
|
Conflicts:
apt-pkg/acquire-item.cc
configure.ac
debian/changelog
doc/apt-verbatim.ent
doc/po/apt-doc.pot
doc/po/de.po
doc/po/es.po
doc/po/fr.po
doc/po/it.po
doc/po/ja.po
doc/po/pt.po
po/ar.po
po/ast.po
po/bg.po
po/bs.po
po/ca.po
po/cs.po
po/cy.po
po/da.po
po/de.po
po/dz.po
po/el.po
po/es.po
po/eu.po
po/fi.po
po/fr.po
po/gl.po
po/hu.po
po/it.po
po/ja.po
po/km.po
po/ko.po
po/ku.po
po/lt.po
po/mr.po
po/nb.po
po/ne.po
po/nl.po
po/nn.po
po/pl.po
po/pt.po
po/pt_BR.po
po/ro.po
po/ru.po
po/sk.po
po/sl.po
po/sv.po
po/th.po
po/tl.po
po/tr.po
po/uk.po
po/vi.po
po/zh_CN.po
po/zh_TW.po
test/integration/test-ubuntu-bug-346386-apt-get-update-paywall
|
|
StringToBool uses strtol() internally to check if the argument is
a number. This function stops when it does not find any more numbers.
So a string like "0ad" (which is a valid packagename) is interpreted
as a "0". The code now checks that the entire string is consumed
not just a part of it. Thanks to Johannes Schauer for raising this
issue.
|
|
This avoid the subtle problem that someone might have a directory
with the same package name as the build-depends he/she is trying
to fetch. Also print a note that the specific file/dir is used.
|
|
Conflicts:
apt-pkg/deb/deblistparser.cc
doc/po/apt-doc.pot
doc/po/de.po
doc/po/es.po
doc/po/fr.po
doc/po/it.po
doc/po/ja.po
doc/po/pl.po
doc/po/pt.po
doc/po/pt_BR.po
po/da.po
po/mr.po
po/vi.po
|
|
|
|
Conflicts:
apt-private/private-install.cc
|
|
The name suggests that it is supposed to substitute a variable with a
value, but we tend to use it in a more liberal replace_all() fashion,
but this breaks if either of the parameters is empty or more importantly
if two "variable" occurrences follow each other directly.
|
|
Conflicts:
debian/changelog
|
|
Reported-By: scan-build
|
|
Removes the 256 fields limit, deals consistently with spaces littered
all over the place and is even a tiny bit faster than before.
Even comes with a bunch of new tests to validate these claims.
|
|
It is not very extensible to have the supported Hashes hardcoded
everywhere and especially if it is part of virtual method names.
It is also possible that a method does not support the 'best' hash
(yet), so we might end up not being able to verify a file even though we
have a common subset of supported hashes. And those are just two of the
cases in which it is handy to have a more dynamic selection.
The downside is that this is a MAJOR API break, but the HashStringList
has a string constructor for compatibility, so with a bit of luck the
few frontends playing with the acquire system directly are okay.
|
|
APT supports more than just one HashString and even allows to enforce
the usage of a specific hash. This class is intended to help with
storage and passing around of the HashStrings.
Git-Dch: Ignore
|
|
|
