| Age | Commit message (Collapse) | Author |
|---|
|
|
|
The by-hash can be configured on a per-hostname basis and a Release
file can indicate that it has by-hash support via a new flag.
The location of the hash now matches the AptByHash spec
|
|
|
|
This implements a apt update schema that get the indexfiles by the
hash instead of the name. The rational is that updates to the archive
servers/mirrors are not atomic so the client may have the previous
version of the Release file when the server updates to a new
Release file and new Packages/Sources/Translations indexes. By
keeping the files around by their hash we can still get the previous
indexfile without a hashsum mismatch.
Enable with APT::Acquire::By-Hash=1
|
|
debian/experimental
Conflicts:
apt-pkg/acquire-item.cc
apt-pkg/acquire-item.h
|
|
pkgAcq{DiffIndex,IndexMerge,pkgAcqBaseIndex, pkgAcqIndex}
|
|
If one of the pkgAcqIndex{,Merge}Diffs fails, they will run
pkgAcqIndex() which needs the IndexTarget/indexRecords data.
So we pass it along.
|
|
Beside being another big API break with hopefully zero fallout in
reality it avoids having the same member and helper code in each and
every subclass.
|
|
It is not very extensible to have the supported Hashes hardcoded
everywhere and especially if it is part of virtual method names.
It is also possible that a method does not support the 'best' hash
(yet), so we might end up not being able to verify a file even though we
have a common subset of supported hashes. And those are just two of the
cases in which it is handy to have a more dynamic selection.
The downside is that this is a MAJOR API break, but the HashStringList
has a string constructor for compatibility, so with a bit of luck the
few frontends playing with the acquire system directly are okay.
|
|
debian/experimental
|
|
bugfix/update-progress-reporting
|
|
debian/experimental
|
|
debian/experimental
|
|
Conflicts:
apt-pkg/cachefilter.h
apt-pkg/contrib/fileutl.cc
apt-pkg/contrib/netrc.h
apt-pkg/deb/debsrcrecords.cc
apt-pkg/init.h
apt-pkg/pkgcache.cc
debian/apt.install.in
debian/changelog
|
|
gcc reports in testcase ./test-bug-596498-trusted-unsigned-repo:
apt-pkg/acquire-item.cc:1059:7: runtime error: load of value 234, which
is not a valid value for type 'bool'
This happens as the bool Verify is initialized only in one of the two
constructors of the pkgAcqIndex class. It isn't a problem through as the
verification controlled by this flag is optional and used to fail early
on garbage files (like network portal pages) instead of later on in the
hashsum verification or while parsing (the then untrusted) file.
Reported-By: gcc-4.9 -fsanitize=undefined
|
|
|
|
used to create a proper pkgAcqIndex() with size information
|
|
fetcher
|
|
with the correct extension
|
|
progress information
|
|
|
|
Otherwise the "WARNING: The following packages cannot be authenticated!"
messages does not include the architecture of the package, so it would
be slightly misinformative.
|
|
Git-Dch: Ignore
Reported-By: gcc -Wsuggest-attribute={pure,const,noreturn}
|
|
Beside being a bit cleaner it hopefully also resolves oddball problems
I have with high levels of parallel jobs.
Git-Dch: Ignore
Reported-By: iwyu (include-what-you-use)
|
|
Reported-By: gcc -Wunused-parameter
Git-Dch: Ignore
|
|
Git-Dch: Ignore
Reported-By: gcc -Wuseless-cast
|
|
Conflicts:
apt-private/private-list.cc
configure.ac
debian/apt.install.in
debian/changelog
|
|
|
|
|
|
With APT::Get::List-Cleanup disabled the ed-style patch files are
lingering in the lists/ directory otherwise. That was kinda okay in the
old none-client-merge as the filename was always the same so it was
constantly overridden, but now with different names for client-merge
quiet a few could pill up on the system and are used by the next call
as it picks them up based on the filename.
|
|
Providing the benefits of both without the downsides :)
(ABI breaks or external dependencies)
For this Anthonys rred is equipped with:
- magic-filename-pickup of patches rather than explicit messages
- use of FileFd instead of FILE* to get on-the-fly uncompress
of the gzip compressed pdiff patches
The acquire code in turn stops checking for apt-file's helper
as our own rred is now clever enough for our needs.
|
|
In 51fc6def77edfb1f429a48e5169519e9e05a759b we limited the amount of
pdiff to be downloaded per index to 20. This was a compromise between
not letting it go overboard (becoming even slower) and not using
bandwidth needlessly. Now that with the POC the speed reason is gone it
makes sense again to download as much files as we possible can via pdiff
to save bandwidth (and possibly even time).
It also avoids problems with the limit in cases we were we deal with a
server merged archieve as this limit assumes a strict patch progression.
|
|
|
|
The idea of pdiffs is to avoid downloading the hole file by patching the
existing index. This works very well, but becomes slow if a lot of
patches needs to be applied to reconstruct an up-to-date index and in
recent years more and more dinstall (or similar) runs are executed
creating more and more pdiffs in the same amount of time, so pdiffs
became less useful.
The solution is simple: Reduce the amount of patches (which are very
small) which need to be applied on top of the index we have available
(which is usually pretty big).
This can be done in two ways: Either merge the patches on the
server-side so that the client has to download only one patch or the
patches are all downloaded and merged on the client-side.
The first needs a client who is doing one step at a time who can also
skip patches if it needs (APT supports this for a long time now).
The later is implemented by this commit, but depends on the server NOT
merging the patches and the patches being in a strict order in which no
patch is skipped.
This is traditionally the case for dak, but other repository creators
support merging – e.g. reprepro (which helpfully adds a flag indicating
that the patches are merged). To support both or even mixes a client
needs more information which isn't available for now.
This POC uses the external diffindex-rred included in apt-file to
do the heavy lifting of merging & applying all patches in one pass,
hence to test this feature apt-file needs to be installed.
|
|
The previous code already did this, this is just being a hell of a lot
more obvious, so that it isn't that easy to break in the future.
Git-Dch: Ignore
|
|
|
|
With a bit of trickery we can reuse the usual infrastructure we have in
place to acquire deb files for the 'download' operation as well, which
gains us authentification check & display, error messages, correct
filenames and "downloads" from the root-owned archives.
|
|
This helps ensure three things:
- each error is reported via ReportMirrorFailure
- if DestFile doesn't exist, do not attempt rename
- renames happen for every error
The last one wasn't the case for Size mismatches, which isn't nice, but
not a exploitable problem per-se as the file isn't picked up and remains
in partial/ where the following download-try will at most take it for a
partial request which fails the hashsum verification later on
Git-Dch: Ignore
|
|
--allow-unauthenticated switches the download to a pre-0.6 system in
which a package can come from any source, rather than that trusted
packages can only come from trusted sources.
To allow this the flag used to set all packages as untrusted, which is a
bit much, so we check now if the package can be acquired via an
untrusted source and only if this is the case set it as untrusted.
As APT nowadays supports setting sources as trusted via a flag in the
sources.list this mode shouldn't be used that much anymore though.
[Note that this is not the patch from the BTS]
Closes: 617690
|
|
|
|
Conflicts:
apt-pkg/tagfile.h
|
|
Conflicts:
apt-pkg/contrib/strutl.cc
apt-pkg/deb/dpkgpm.cc
configure.ac
debian/changelog
doc/po/apt-doc.pot
po/apt-all.pot
po/ar.po
po/ast.po
po/bg.po
po/bs.po
po/ca.po
po/cs.po
po/cy.po
po/da.po
po/de.po
po/dz.po
po/el.po
po/es.po
po/eu.po
po/fi.po
po/fr.po
po/gl.po
po/hu.po
po/it.po
po/ja.po
po/km.po
po/ko.po
po/ku.po
po/lt.po
po/mr.po
po/nb.po
po/ne.po
po/nl.po
po/nn.po
po/pl.po
po/pt.po
po/pt_BR.po
po/ro.po
po/ru.po
po/sk.po
po/sl.po
po/sv.po
po/th.po
po/tl.po
po/uk.po
po/vi.po
po/zh_CN.po
po/zh_TW.po
test/integration/framework
test/integration/test-bug-602412-dequote-redirect
test/integration/test-ubuntu-bug-346386-apt-get-update-paywall
test/interactive-helper/aptwebserver.cc
test/interactive-helper/makefile
|
|
The constructors of our (clear)sign-acquire-items move a pre-existent
file for error-recovery away, which gets restored or discarded later as
the acquire progresses, but --print-uris never really starts the
acquire process, so the files aren't restored (as they should).
To fix this both get a destructor which checks for signs of acquire
doing anything and if it hasn't the file is restored.
Note that these virtual destructors theoretically break the API, but
only with classes extending the sign-acquire-items and nobody does this,
as it would be insane for library users to fiddle with Acquire
internals – and these classes are internals.
Closes: 719263
|
|
|
|
On CD-ROMs Translation-* files are only in compressed form included in
the Release file. This used to work while we had no record of
Translation-* files in the Release file at all as APT would have just
guessed the (compressed) filename and accepted it (unchecked), but now
that it checks for the presents of entries and if it finds records it
expects the uncompressed to be verifiable.
This commit relaxes this requirement again to fix the regression.
We are still secure "enough" as we can validate the compressed file we have
downloaded, so we don't loose anything by not requiring a hashsum for
the uncompressed files to double-check them.
Closes: 717665
|
|
Before we download the 'new' InRelease file the old file will be moved
out of the way with the name 'foobar_InRelease.reverify', so if no
partial file for the 'new' file exists take the modification time from
this reverify file, so that if we get an IMS hit for the InRelease file
we can move back the reverify file as new file rather than downloading
the 'new' file even though we already have it.
We do the same for Release files and this happened to work until the
reverify renaming was corrected for InRelease files.
|
|
|
|
translations. Closes: #705087
|
|
|
|
|
