summaryrefslogtreecommitdiff
path: root/apt-pkg/acquire-item.cc
AgeCommit message (Collapse)Author
2013-12-13implement POC client-side merging of pdiffs via apt-fileDavid Kalnischkies
The idea of pdiffs is to avoid downloading the hole file by patching the existing index. This works very well, but becomes slow if a lot of patches needs to be applied to reconstruct an up-to-date index and in recent years more and more dinstall (or similar) runs are executed creating more and more pdiffs in the same amount of time, so pdiffs became less useful. The solution is simple: Reduce the amount of patches (which are very small) which need to be applied on top of the index we have available (which is usually pretty big). This can be done in two ways: Either merge the patches on the server-side so that the client has to download only one patch or the patches are all downloaded and merged on the client-side. The first needs a client who is doing one step at a time who can also skip patches if it needs (APT supports this for a long time now). The later is implemented by this commit, but depends on the server NOT merging the patches and the patches being in a strict order in which no patch is skipped. This is traditionally the case for dak, but other repository creators support merging – e.g. reprepro (which helpfully adds a flag indicating that the patches are merged). To support both or even mixes a client needs more information which isn't available for now. This POC uses the external diffindex-rred included in apt-file to do the heavy lifting of merging & applying all patches in one pass, hence to test this feature apt-file needs to be installed.
2013-12-13query an empty pkgAcqIndexDiffs if index is up-to-dateDavid Kalnischkies
The previous code already did this, this is just being a hell of a lot more obvious, so that it isn't that easy to break in the future. Git-Dch: Ignore
2013-10-09Merge remote-tracking branch 'mvo/feature/limit-default-pdiffs' into debian/sidMichael Vogt
2013-10-03use pkgAcqArchive in 'download' for proper errorsDavid Kalnischkies
With a bit of trickery we can reuse the usual infrastructure we have in place to acquire deb files for the 'download' operation as well, which gains us authentification check & display, error messages, correct filenames and "downloads" from the root-owned archives.
2013-10-03refactor onError relabeling of DestFile as '.FAILED'David Kalnischkies
This helps ensure three things: - each error is reported via ReportMirrorFailure - if DestFile doesn't exist, do not attempt rename - renames happen for every error The last one wasn't the case for Size mismatches, which isn't nice, but not a exploitable problem per-se as the file isn't picked up and remains in partial/ where the following download-try will at most take it for a partial request which fails the hashsum verification later on Git-Dch: Ignore
2013-09-26pkg from only trusted sources keeps being trustedDavid Kalnischkies
--allow-unauthenticated switches the download to a pre-0.6 system in which a package can come from any source, rather than that trusted packages can only come from trusted sources. To allow this the flag used to set all packages as untrusted, which is a bit much, so we check now if the package can be acquired via an untrusted source and only if this is the case set it as untrusted. As APT nowadays supports setting sources as trusted via a flag in the sources.list this mode shouldn't be used that much anymore though. [Note that this is not the patch from the BTS] Closes: 617690
2013-08-26set Acquire::PDiffs::FileLimit to 20 to avoid needless huge fetchesMichael Vogt
2013-08-22Merge remote-tracking branch 'mvo/bugfix/coverity' into debian/sidMichael Vogt
Conflicts: apt-pkg/tagfile.h
2013-08-12fix: --print-uris removes authenticationDavid Kalnischkies
The constructors of our (clear)sign-acquire-items move a pre-existent file for error-recovery away, which gets restored or discarded later as the acquire progresses, but --print-uris never really starts the acquire process, so the files aren't restored (as they should). To fix this both get a destructor which checks for signs of acquire doing anything and if it hasn't the file is restored. Note that these virtual destructors theoretically break the API, but only with classes extending the sign-acquire-items and nobody does this, as it would be insane for library users to fiddle with Acquire internals – and these classes are internals. Closes: 719263
2013-08-12some more coverity fixesMichael Vogt
2013-07-25pick up Translation-* even if only compressed availableDavid Kalnischkies
On CD-ROMs Translation-* files are only in compressed form included in the Release file. This used to work while we had no record of Translation-* files in the Release file at all as APT would have just guessed the (compressed) filename and accepted it (unchecked), but now that it checks for the presents of entries and if it finds records it expects the uncompressed to be verifiable. This commit relaxes this requirement again to fix the regression. We are still secure "enough" as we can validate the compressed file we have downloaded, so we don't loose anything by not requiring a hashsum for the uncompressed files to double-check them. Closes: 717665
2013-06-20do not redownload unchanged InRelease filesDavid Kalnischkies
Before we download the 'new' InRelease file the old file will be moved out of the way with the name 'foobar_InRelease.reverify', so if no partial file for the 'new' file exists take the modification time from this reverify file, so that if we get an IMS hit for the InRelease file we can move back the reverify file as new file rather than downloading the 'new' file even though we already have it. We do the same for Release files and this happened to work until the reverify renaming was corrected for InRelease files.
2013-04-10Fix English spelling error in a message ('A error'). Unfuzzybubulle@debian.org
translations. Closes: #705087
2013-04-08merged bundle from davidMichael Vogt
2013-04-02merged lp:~mvo/apt/fix-inrelease5Michael Vogt
2013-03-15* apt-pkg/acquire-item.cc:David Kalnischkies
- keep the last good InRelease file around just as we do it with Release.gpg in case the new one we download isn't good for us
2013-03-10various simple changes to fix cppcheck warningsDavid Kalnischkies
2013-01-14ensure sha512 is really used when available (thanks to Tyler Hicks )Michael Vogt
2012-03-06add Debug::pkgAcqArchive::NoQueue to disable package downloadingDavid Kalnischkies
2012-03-04* apt-pkg/acquire-item.cc:David Kalnischkies
- remove 'old' InRelease file if we can't get a new one before proceeding with Release.gpg to avoid the false impression of a still trusted repository by a (still present) old InRelease file. Thanks to Simon Ruderich for reporting this issue! (CVE-2012-0214) Effected are all versions >= 0.8.11 Possible attack summary: - Attacker needs to find a user which has run at least one successful 'apt-get update' against an archive providing InRelease files. - Create a Packages file with his preferred content. - Attacker then prevents the download of InRelease, Release and Release.gpg (alternatively he creates a valid Release file and sends this, the other two files need to be missing either way). - User updates against this, getting the modified Packages file without any indication of being unsigned (beside the "Ign InRelease" and "Ign Release.gpg" in the output of 'apt-get update'). => deb files from this source are considered 'trusted' (and therefore the user isn't asked for an additional confirmation before install)
2012-02-18use pdiff for Translation-* files if available (Closes: #657902)David Kalnischkies
Beware: pdiffs for Translation-* are only acquired if their availability is advertised in the Release file.
2012-02-18* apt-pkg/acquire-item.cc:David Kalnischkies
- drop support for i18n/Index file (introduced in 0.8.11) and use the Release file instead to get the Translations (Closes: #649314) * ftparchive/writer.cc: - add 'Translation-*' to the default patterns i18n/Index was never used outside debian - and even here it isn't used consistently as only 'main' has such a file. As the Release file now includes the Translation-* files we therefore drop support for i18n/Index. A version supporting it was never part of a debian release and still supporting it would mean that we get 99% of the time a 404 as response to the request anyway and confuse archive maintainers who want to provide all files APT tries to acquire.
2011-12-17try to avoid direct usage of .Fd() if possible and do read()s and coDavid Kalnischkies
on the FileFd instead
2011-09-19use forward declaration in headers if possible instead of includesDavid Kalnischkies
2011-09-13merge with debian/sidDavid Kalnischkies
2011-09-13merge with debian/experimentalDavid Kalnischkies
2011-09-13reorder includes: add <config.h> if needed and include it at firstDavid Kalnischkies
2011-08-22* apt-pkg/acquire-item.cc:David Kalnischkies
- if no Release.gpg file is found try to verify with hashes, but do not fail if a hash can't be found
2011-08-15merged from the debian-sid branchMichael Vogt
2011-08-15merged fixes from lp:~mvo/apt/mvoMichael Vogt
2011-08-11cppcheck complains about some possible speed improvements which could beDavid Kalnischkies
done on the mirco-optimazation level, so lets fix them: (performance) Possible inefficient checking for emptiness. (performance) Prefer prefix ++/-- operators for non-primitive types.
2011-08-08fix crash when P.Arch() was used but the cache got remappedMichael Vogt
2011-08-08apt-pkg/acquire-item.cc: add more debug outputMichael Vogt
2011-08-05* test/integration/test-hashsum-verification:Michael Vogt
- add regression test for hashsum verification * apt-pkg/acquire-item.cc: - if no Release.gpg file is found, still load the hashes for verification (closes: #636314) and add test
2011-08-05* apt-pkg/acquire-item.cc:Michael Vogt
- if no Release.gpg file is found, still load the hashes for verification (closes: #636314) and add test
2011-08-05apt-pkg/acquire-item.cc: always init VerifyMichael Vogt
2011-08-05* apt-pkg/acquire-item.{cc,h}:Michael Vogt
- do not check for a "Package" tag in optional index targets like the translations index
2011-07-28* [ABI break] apt-pkg/acquire-item.{cc,h}:Michael Vogt
- cleanup around OptionalIndexTarget and SubIndexTarget
2011-07-15merged from http://bzr.debian.org/bzr/apt/apt/debian-sidMichael Vogt
2011-07-05* apt-pkg/acquire*.{cc,h}:David Kalnischkies
- try even harder to support really big files in the fetcher by converting (hopefully) everything to 'long long' (Closes: #632271)
2011-07-01apt-pkg/acquire-item.cc: improve error message for valid-untilMichael Vogt
2011-06-29merge lp:~mvo/apt/abi-breakMichael Vogt
2011-06-08merge lp:~mvo/apt/sha512-template to add support for sha512Michael Vogt
2011-05-31apt-pkg/acquire-item.cc: only test packages file for correctness if its not ↵Michael Vogt
empty (its ok to have empty packages files)
2011-05-30Reject files known to be invalid (LP: #346386) (Closes: #627642)Julian Andres Klode
2011-05-30apt-pkg/acquire-item.cc: Reject files known to be invalid (LP: #346386) ↵Julian Andres Klode
(Closes: #195301) This commit deals with the following cases: - First section of index file (Packages,Sources,Translation) without Package field - Signed release files without GPG data (NODATA) - i18n/Index files without hash sums Handling unsigned Release files is more complicated, and the example code using indexRecords is disabled as it can reject correct Release files without hashes. How we can reliably check unsigned Release files is another question, and not urgent anyway, as it should have no dramatic effect (we could check that it is a valid RFC-822 section, but that's a bit too long to write)
2011-04-26* apt-pkg/acquire-item.cc:Ben Finney
- apply fix for poorly worded 'locate file' error message from Ben Finney, thanks! (Closes: #623171)
2011-04-16apt-pkg/acquire-item.cc: Only try to rename existing Release files (Closes: ↵Julian Andres Klode
#622912)
2011-04-08* apt-pkg/acquire-item.cc:Julian Andres Klode
- Use Release files even if they cannot be verified (LP: #704595)
2011-04-04merged from lp:~donkult/apt/sidMichael Vogt