| Age | Commit message (Collapse) | Author |
|---|
|
The previous implementation was still a bit unstable in terms of failing
at times. Lets try if we have more luck with this one.
Git-Dch: Ignore
|
|
Our error reporting is historically grown into some kind of mess.
A while ago I implemented stacking for the global error which is used in
this commit now to wrap calls to functions which do not report (all)
errors via return, so that only failures in those calls cause a failure
to propergate down the chain rather than failing if anything
(potentially totally unrelated) has failed at some point in the past.
This way we can avoid stopping the entire acquire process just because a
single source produced an error for example. It also means that after
the acquire process the cache is generated – even if the acquire
process had failures – as we still have the old good data around we can and
should generate a cache for (again).
There are probably more instances of this hiding, but all these looked
like the easiest to work with and fix with reasonable (aka net-positive)
effects.
|
|
clang detects that fd isn't set in the ReadWrite case – just that this
is supposed to be catched earlier in this method already, but it doesn't
hurt to make it explicit here as well and clang is happy, too.
Git-Dch: Ignore
|
|
Pipes and such have no good Size value, but we still want to copy from
it maybe and we don't really need size as we can just as well read as
long as we get data out of a file to copy it.
Git-Dch: Ignore
|
|
Git-Dch: Ignore
|
|
Initializing a random number generator with the time since epoch could
be good enough, but reaches its limits in test code as the 100
iterations might very well happen in the same second and hence the seed
number is always the same… clock() has a way lower resolution so it
changes more often and not unimportant: If many users start the update
at the same time it isn't to unlikely the SRV record will be ordered in
the same second choosing the same for them all, but it seems less likely
that the exact same clock() time has passed for them.
And if I have to touch this, lets change a few other things as well to
make me and/or compilers a bit happier (clang complained about the usage
of a GNU extension in the testcase for example).
|
|
We want to declare some hashes as not enough for security, so that a
user will need --allow-unauthenticated or similar to get data secured
only by those hashes, but we can still us these hashes for integrity
checks if we got them.
|
|
The md5sum hash is broken since some time and we should no longer
consider it a usable hash. Also update the tests to reflect this.
|
|
Various smaller improvements so that the check deals better with already
downloaded files, relative paths and other things.
Git-Dch: Ignore
|
|
Using libpam-tmpdir caused us to create our download tmp directory in
root's private tmp before changing to _apt, which wouldn't have access
to it.
By extending our GetTempDir method with an optional wrapper changing the
effective user, we can test if a given user can access the directory and
ignore TMPDIR if not instead of ignoring TMPDIR completely.
Closes: 797270
|
|
We dup() the file descriptor when opening compressed files, so we
always need to close the dup()ed one. Furthermore, not unsetting
the d-pointer causes issues when running OpenDescriptor() multiple
times on the same file descriptor.
|
|
Also add "Debug::Acquire::SrvRecs" debug option and the option
"Acquire::EnableSrvRecods" to allow disabling this lookup.
|
|
|
|
|
|
feature/srv-records
|
|
Conflicts:
cmdline/apt-helper.cc
cmdline/makefile
|
|
Gbp-Dch: ignore
|
|
Closes: #762758
|
|
|
|
More standardization
|
|
Switch to std::unique_ptr, as this is safer than SPtr.
|
|
Gbp-Dch: ignore
Reported-By: g++ -Wsuggest-override
Thanks: g++ -Wsuggest-override
|
|
This significantly reduces the number of files that have to be closed
and seems to be faster, despite the additional reads.
On systems where /proc/self/fd is not available, we fallback to the
old code that closes all file descriptors >= 3.
Closes: #764204
|
|
We changed an aweful lot of stuff, so 5.0 is properly better than 4.X as
a semantic version and as we are at it lets add some trivial symbol
versioning as well: We just mark all exported symbols with the same
version for now. This isn't really the proper thing to do as if we add
symbols in later versions (with the same abi) they will get the same
symbols version, but our .symbols file will protect us from the problems
arising from this as it will ensure that a package acutally depends on a
version of the abi high enough to include the symbol.
|
|
Trade deduplication of code for a bunch of new virtuals, so it is
actually visible how the different indexes behave cleaning up the
interface at large in the process.
Git-Dch: Ignore
|
|
We aren't and we will not be really compatible again with the previous
stable abi, so lets drop these markers (which never made it into a
released version) for good as they have outlived their intend already.
Git-Dch: Ignore
|
|
C++11 adds the 'override' specifier to mark that a method is overriding
a base class method and error out if not. We hide it in the APT_OVERRIDE
macro to ensure that we keep compiling in pre-c++11 standards.
Reported-By: clang-modernize -add-override -override-macros
Git-Dch: Ignore
|
|
By further abstracting the iterator templates we can wrap the reverse
iterators of the wrapped containers and share code in a way that
iterator creating is now more template intensive, but shorter in code.
Git-Dch: Ignore
|
|
Limits which key(s) can be used to sign a repository. Not immensely useful
from a security perspective all by itself, but if the user has
additional measures in place to confine a repository (like pinning) an
attacker who gets the key for such a repository is limited to its
potential and can't use the key to sign its attacks for an other (maybe
less limited) repository… (yes, this is as weak as it sounds, but having
the capability might come in handy for implementing other stuff later).
|
|
Various small leaks here and there. Nothing particularily big, but still
good to fix. Found by the sanitizers while running our testcases.
Reported-By: gcc -fsanitize
Git-Dch: Ignore
|
|
More warnings are always better.
Git-Dch: Ignore
|
|
Doing this disables the implicit copy assignment operator (among others)
which would cause hovac if used on the classes as it would just copy the
pointer, not the data the d-pointer points to. For most of the classes
we don't need a copy assignment operator anyway and in many classes it
was broken before as many contain a pointer of some sort.
Only for our Cacheset Container interfaces we define an explicit copy
assignment operator which could later be implemented to copy the data
from one d-pointer to the other if we need it.
Git-Dch: Ignore
|
|
Some of them modify the ABI, but given that we prepare a big one
already, these few hardly count for much.
Git-Dch: Ignore
|
|
It is a rather strange sight that index items use SiteOnly which strips
the Path, while e.g. deb files are downloaded with NoUserPassword which
does not. Important to note here is that for the file transport Path is
pretty important as there is no Host which would be displayed by Site,
which always resulted in "interesting" unspecific errors for "file:".
Adding a 'middle' ground between the two which does show the Path but
potentially modifies it (it strips a pending / at the end if existing)
solves this "file:" issue, syncs the output and in the end helps to
identify which file is meant exactly in progress output and co as a
single site can have multiple repositories in different paths.
|
|
First pass at making the acquire system capable of downloading files
based on configuration rather than hardcoded entries. It is now possible
to instruct 'deb' and 'deb-src' sources.list lines to download more than
just Packages/Translation-* and Sources files. Details on how to do that
can be found in the included documentation file.
|
|
At the moment we only have hashes for the uncompressed pdiff files, but
via the new '$HASH-Download' field in the .diff/Index hashes can be
provided for the .gz compressed pdiff file, which apt will pick up now
and use to verify the download. Now, we "just" need a buy in from the
creators of repositories…
|
|
Having every item having its own code to verify the file(s) it handles
is an errorprune process and easy to break, especially if items move
through various stages (download, uncompress, patching, …). With a giant
rework we centralize (most of) the verification to have a better
enforcement rate and (hopefully) less chance for bugs, but it breaks the
ABI bigtime in exchange – and as we break it anyway, it is broken even
harder.
It shouldn't effect most frontends as they don't deal with the acquire
system at all or implement their own items, but some do and will need to
be patched (might be an opportunity to use apt on-board material).
The theory is simple: Items implement methods to decide if hashes need to
be checked (in this stage) and to return the expected hashes for this
item (in this stage). The verification itself is done in worker message
passing which has the benefit that a hashsum error is now a proper error
for the acquire system rather than a Done() which is later revised to a
Failed().
|
|
Conflicts:
apt-pkg/pkgcache.h
debian/changelog
methods/https.cc
methods/server.cc
test/integration/test-apt-download-progress
|
|
Conflicts:
apt-pkg/deb/dpkgpm.cc
|
|
It isn't used much compared to what the methodname suggests, but in the
remaining uses it can't hurt to check more than strictly necessary by
calculating and verifying with all hashes we can compare with rather
than "just" the best known hash.
|
|
Git-Dch: Ignore
|
|
|
|
Conflicts:
apt-pkg/acquire-item.cc
cmdline/apt-key.in
methods/https.cc
test/integration/test-apt-key
test/integration/test-multiarch-foreign
|
|
Its a bit unpredictable which permissons and owners we will encounter on
a CD-ROM (or a USB stick, as apt-cdrom is responsible for those too),
so we have to ensure in this codepath as well that everything is nicely
setup without waiting for a 'apt-get update' to fix up the (potential)
mess.
|
|
Methods get told which hashes are expected by the acquire system, which
means we can use this list to restrict what we calculate in the methods
as any extra we are calculating is wasted effort as we can't compare it
with anything anyway.
Adding support for a new hash algorithm is therefore 'free' now and if a
algorithm is no longer provided in a repository for a file, we
automatically stop calculating it.
In practice this results in a speed-up in Debian as we don't have SHA512
here (so far), so we practically stop calculating it.
|
|
Git-Dch: Ignore
|
|
Working with strings c-style is complicated and error-prune,
so by converting to c++ style we gain some simplicity and
avoid buffer overflows by later extensions.
Git-Dch: Ignore
|
|
g++-5 generates a slightly broken libapt which doesn't split
architecture configurations correctly resulting in e.g. Packages files
requested for the bogus architecture 'amd64,i386' instead of for amd64
and i386.
The reason is an incorrectly applied attribute marking the function as
const, while functions with pointer arguments are not allowed to be
declared as such (note that char& is a char* in disguise). Demoting the
attribute to pure fixes this issue – better would be dropping the & from
char but that is an API change…
Neither earlier g++ versions nor clang use this attribute to generate
broken code, so we don't need a rebuild of dependencies or anything and
g++-5 isn't even included in jessie, but the effect is so strange and
apt popular enough to consider avoiding this problem anyhow.
|
|
Git-Dch: Ignore
|
|
This used to work before we implemented a stricter commandline parser
and e.g. the dd-schroot-cmd command constructs commandlines like this.
Reported-By: Helmut Grohne
|
