| Age | Commit message (Collapse) | Author |
|---|
|
The idea of pdiffs is to avoid downloading the hole file by patching the
existing index. This works very well, but becomes slow if a lot of
patches needs to be applied to reconstruct an up-to-date index and in
recent years more and more dinstall (or similar) runs are executed
creating more and more pdiffs in the same amount of time, so pdiffs
became less useful.
The solution is simple: Reduce the amount of patches (which are very
small) which need to be applied on top of the index we have available
(which is usually pretty big).
This can be done in two ways: Either merge the patches on the
server-side so that the client has to download only one patch or the
patches are all downloaded and merged on the client-side.
The first needs a client who is doing one step at a time who can also
skip patches if it needs (APT supports this for a long time now).
The later is implemented by this commit, but depends on the server NOT
merging the patches and the patches being in a strict order in which no
patch is skipped.
This is traditionally the case for dak, but other repository creators
support merging – e.g. reprepro (which helpfully adds a flag indicating
that the patches are merged). To support both or even mixes a client
needs more information which isn't available for now.
This POC uses the external diffindex-rred included in apt-file to
do the heavy lifting of merging & applying all patches in one pass,
hence to test this feature apt-file needs to be installed.
|
|
The previous code already did this, this is just being a hell of a lot
more obvious, so that it isn't that easy to break in the future.
Git-Dch: Ignore
|
|
|
|
With a bit of trickery we can reuse the usual infrastructure we have in
place to acquire deb files for the 'download' operation as well, which
gains us authentification check & display, error messages, correct
filenames and "downloads" from the root-owned archives.
|
|
This helps ensure three things:
- each error is reported via ReportMirrorFailure
- if DestFile doesn't exist, do not attempt rename
- renames happen for every error
The last one wasn't the case for Size mismatches, which isn't nice, but
not a exploitable problem per-se as the file isn't picked up and remains
in partial/ where the following download-try will at most take it for a
partial request which fails the hashsum verification later on
Git-Dch: Ignore
|
|
--allow-unauthenticated switches the download to a pre-0.6 system in
which a package can come from any source, rather than that trusted
packages can only come from trusted sources.
To allow this the flag used to set all packages as untrusted, which is a
bit much, so we check now if the package can be acquired via an
untrusted source and only if this is the case set it as untrusted.
As APT nowadays supports setting sources as trusted via a flag in the
sources.list this mode shouldn't be used that much anymore though.
[Note that this is not the patch from the BTS]
Closes: 617690
|
|
|
|
Conflicts:
apt-pkg/tagfile.h
|
|
The constructors of our (clear)sign-acquire-items move a pre-existent
file for error-recovery away, which gets restored or discarded later as
the acquire progresses, but --print-uris never really starts the
acquire process, so the files aren't restored (as they should).
To fix this both get a destructor which checks for signs of acquire
doing anything and if it hasn't the file is restored.
Note that these virtual destructors theoretically break the API, but
only with classes extending the sign-acquire-items and nobody does this,
as it would be insane for library users to fiddle with Acquire
internals – and these classes are internals.
Closes: 719263
|
|
|
|
On CD-ROMs Translation-* files are only in compressed form included in
the Release file. This used to work while we had no record of
Translation-* files in the Release file at all as APT would have just
guessed the (compressed) filename and accepted it (unchecked), but now
that it checks for the presents of entries and if it finds records it
expects the uncompressed to be verifiable.
This commit relaxes this requirement again to fix the regression.
We are still secure "enough" as we can validate the compressed file we have
downloaded, so we don't loose anything by not requiring a hashsum for
the uncompressed files to double-check them.
Closes: 717665
|
|
Before we download the 'new' InRelease file the old file will be moved
out of the way with the name 'foobar_InRelease.reverify', so if no
partial file for the 'new' file exists take the modification time from
this reverify file, so that if we get an IMS hit for the InRelease file
we can move back the reverify file as new file rather than downloading
the 'new' file even though we already have it.
We do the same for Release files and this happened to work until the
reverify renaming was corrected for InRelease files.
|
|
translations. Closes: #705087
|
|
|
|
|
|
- keep the last good InRelease file around just as we do it with
Release.gpg in case the new one we download isn't good for us
|
|
|
|
|
|
|
|
- remove 'old' InRelease file if we can't get a new one before
proceeding with Release.gpg to avoid the false impression of a still
trusted repository by a (still present) old InRelease file.
Thanks to Simon Ruderich for reporting this issue! (CVE-2012-0214)
Effected are all versions >= 0.8.11
Possible attack summary:
- Attacker needs to find a user which has run at least one successful
'apt-get update' against an archive providing InRelease files.
- Create a Packages file with his preferred content.
- Attacker then prevents the download of InRelease, Release and
Release.gpg (alternatively he creates a valid Release file and sends
this, the other two files need to be missing either way).
- User updates against this, getting the modified Packages file without
any indication of being unsigned (beside the "Ign InRelease" and
"Ign Release.gpg" in the output of 'apt-get update').
=> deb files from this source are considered 'trusted' (and therefore the
user isn't asked for an additional confirmation before install)
|
|
Beware: pdiffs for Translation-* are only acquired if their
availability is advertised in the Release file.
|
|
- drop support for i18n/Index file (introduced in 0.8.11) and use
the Release file instead to get the Translations (Closes: #649314)
* ftparchive/writer.cc:
- add 'Translation-*' to the default patterns
i18n/Index was never used outside debian - and even here it isn't used
consistently as only 'main' has such a file. As the Release file now
includes the Translation-* files we therefore drop support for i18n/Index.
A version supporting it was never part of a debian release and still
supporting it would mean that we get 99% of the time a 404 as response
to the request anyway and confuse archive maintainers who want to
provide all files APT tries to acquire.
|
|
on the FileFd instead
|
|
|
|
|
|
|
|
|
|
- if no Release.gpg file is found try to verify with hashes,
but do not fail if a hash can't be found
|
|
|
|
|
|
done on the mirco-optimazation level, so lets fix them:
(performance) Possible inefficient checking for emptiness.
(performance) Prefer prefix ++/-- operators for non-primitive types.
|
|
|
|
|
|
- add regression test for hashsum verification
* apt-pkg/acquire-item.cc:
- if no Release.gpg file is found, still load the hashes for
verification (closes: #636314) and add test
|
|
- if no Release.gpg file is found, still load the hashes for
verification (closes: #636314) and add test
|
|
|
|
- do not check for a "Package" tag in optional index targets
like the translations index
|
|
- cleanup around OptionalIndexTarget and SubIndexTarget
|
|
|
|
- try even harder to support really big files in the fetcher by
converting (hopefully) everything to 'long long' (Closes: #632271)
|
|
|
|
|
|
|
|
empty (its ok to have empty packages files)
|
|
|
|
(Closes: #195301)
This commit deals with the following cases:
- First section of index file (Packages,Sources,Translation) without Package field
- Signed release files without GPG data (NODATA)
- i18n/Index files without hash sums
Handling unsigned Release files is more complicated, and the example code
using indexRecords is disabled as it can reject correct Release files
without hashes. How we can reliably check unsigned Release files is
another question, and not urgent anyway, as it should have no
dramatic effect (we could check that it is a valid RFC-822 section,
but that's a bit too long to write)
|
|
- apply fix for poorly worded 'locate file' error message from
Ben Finney, thanks! (Closes: #623171)
|
|
#622912)
|
|
- Use Release files even if they cannot be verified (LP: #704595)
|
|
|
