summaryrefslogtreecommitdiff
path: root/apt-pkg/acquire-item.cc
AgeCommit message (Collapse)Author
2015-08-01hide first pdiff merge failure debug messageDavid Kalnischkies
The siblings of this message are all guarded as debug messages, just this one had it missing and subsequently causes display issues if triggered, which, given that clog is an alias for cerr, end up on stderr and therefore are reported as problems by tools only showing the stderr like our own cron script. [Backport of f4c7a238f4c29ac9b1e1172f103ab7dec5c5807d] Closes: 793444
2015-04-07avoid depends on std::string implementation for pkgAcquire::Item::ModeDavid Kalnischkies
In /experimental this is resolved by deprecating Mode and moving to a new std::string, but that breaks ABI of course, so that was out of question. We can't change to a malloc/free style c-string either as Mode is public and hence a library user could be setting this as well. std::string implementors actually helped us out here with copy-on-write which means that while the variable "obviously" runs out of scope here, in reality you get the correct result as the string we work with here comes from the configuration in which it is still valid. Such a dependency on magic is bad of course, but its still interesting that only python3 seems to have an issue with it… With some silly explicit if-else assigning we can sidestep this issue while retaining the same output for 99.99% of all users (= noone actually configures additional compression algorithms which are also provided by repositories…), but even for these 0.01% its just a small change in the display as Mode can not be used for anything else. Example: apt/aptitude uses it in its 'update' implementations in the one-line progress at the bottom for specific items. Closes: 781858
2014-10-15don't cleanup cdrom files in apt-get updateDavid Kalnischkies
Regression from merging 801745284905e7962aa77a9f37a6b4e7fcdc19d0 and b0f4b486e6850c5f98520ccf19da71d0ed748ae4. While fine by itself, merged the part fixing the filename is skipped if a cdrom source is encountered, so that our list-cleanup removes what seems to be orphaned files. Closes: 765458
2014-09-26Merge remote-tracking branch 'donkult/feature/generalize-gzipindex' into ↵Michael Vogt
debian/sid
2014-09-21Fix regression for cdrom: sources from latest security updateMichael Vogt
Skip a reverify for cdrom: sources. The reverify step is actually harmful here because the apt-cdrom add code uses the indexcopy.cc which will "normalize" the Packages file from the cdrom when it writes it to the local disk. This leads to changing the "MD5sum" field (notice the lower case "s") on the cdrom Packages file to a "MD5Sum" field on the local file in /var/lib/apt/lists. Which of course alters the hash and makes apt fail to reverify the file.
2014-09-21generalize Acquire::GzipIndexMichael Vogt
2014-09-17improve test for commit daff4aMichael Vogt
2014-09-17Fix regression for file:/// uris from CVE-2014-0487Michael Vogt
Do not run ReverifyAfterIMS() for local file URIs as this will causes apt to mess around in the file:/// uri space. This is wrong in itself, but it will also cause a incorrect verification failure when the archive and the lists directory are on different partitions as rename().
2014-09-16SECURITY UPDATE for CVE-2014-{0488,0487,0489}Michael Vogt
incorrect invalidating of unauthenticated data (CVE-2014-0488) incorect verification of 304 reply (CVE-2014-0487) incorrect verification of Acquire::Gzip indexes (CVE-2014-0489)
2014-07-17apt-pkg/acquire-item.cc: make pkgAcqDiffIndex more uniformMichael Vogt
2014-07-03Try not to parse invalid translation files (LP: #756317)Michael Vogt
2014-04-28initialize Verify in second pkgAcqIndex constructorDavid Kalnischkies
gcc reports in testcase ./test-bug-596498-trusted-unsigned-repo: apt-pkg/acquire-item.cc:1059:7: runtime error: load of value 234, which is not a valid value for type 'bool' This happens as the bool Verify is initialized only in one of the two constructors of the pkgAcqIndex class. It isn't a problem through as the verification controlled by this flag is optional and used to fail early on garbage files (like network portal pages) instead of later on in the hashsum verification or while parsing (the then untrusted) file. Reported-By: gcc-4.9 -fsanitize=undefined
2014-03-13use the pretty fullname of a pkg as download desciptionDavid Kalnischkies
Otherwise the "WARNING: The following packages cannot be authenticated!" messages does not include the architecture of the package, so it would be slightly misinformative.
2014-03-13follow method attribute suggestions by gccDavid Kalnischkies
Git-Dch: Ignore Reported-By: gcc -Wsuggest-attribute={pure,const,noreturn}
2014-03-13cleanup headers and especially #includes everywhereDavid Kalnischkies
Beside being a bit cleaner it hopefully also resolves oddball problems I have with high levels of parallel jobs. Git-Dch: Ignore Reported-By: iwyu (include-what-you-use)
2014-03-13warning: unused parameter ‘foo’ [-Wunused-parameter]David Kalnischkies
Reported-By: gcc -Wunused-parameter Git-Dch: Ignore
2014-03-13warning: useless cast to type A [-Wuseless-cast]David Kalnischkies
Git-Dch: Ignore Reported-By: gcc -Wuseless-cast
2014-02-27initial version of apt-helperMichael Vogt
2014-02-22Fix typos in documentation (codespell)Michael Vogt
2014-02-10always cleanup patchfiles at the end of rred callDavid Kalnischkies
With APT::Get::List-Cleanup disabled the ed-style patch files are lingering in the lists/ directory otherwise. That was kinda okay in the old none-client-merge as the filename was always the same so it was constantly overridden, but now with different names for client-merge quiet a few could pill up on the system and are used by the next call as it picks them up based on the filename.
2014-01-15integrate Anthonys rred with POC for client-side mergeDavid Kalnischkies
Providing the benefits of both without the downsides :) (ABI breaks or external dependencies) For this Anthonys rred is equipped with: - magic-filename-pickup of patches rather than explicit messages - use of FileFd instead of FILE* to get on-the-fly uncompress of the gzip compressed pdiff patches The acquire code in turn stops checking for apt-file's helper as our own rred is now clever enough for our needs.
2014-01-15reenable unlimited pdiff files downloadDavid Kalnischkies
In 51fc6def77edfb1f429a48e5169519e9e05a759b we limited the amount of pdiff to be downloaded per index to 20. This was a compromise between not letting it go overboard (becoming even slower) and not using bandwidth needlessly. Now that with the POC the speed reason is gone it makes sense again to download as much files as we possible can via pdiff to save bandwidth (and possibly even time). It also avoids problems with the limit in cases we were we deal with a server merged archieve as this limit assumes a strict patch progression.
2014-01-15correct IndexDiff vs DiffIndex in Debug outputAnthony Towns
2013-12-13implement POC client-side merging of pdiffs via apt-fileDavid Kalnischkies
The idea of pdiffs is to avoid downloading the hole file by patching the existing index. This works very well, but becomes slow if a lot of patches needs to be applied to reconstruct an up-to-date index and in recent years more and more dinstall (or similar) runs are executed creating more and more pdiffs in the same amount of time, so pdiffs became less useful. The solution is simple: Reduce the amount of patches (which are very small) which need to be applied on top of the index we have available (which is usually pretty big). This can be done in two ways: Either merge the patches on the server-side so that the client has to download only one patch or the patches are all downloaded and merged on the client-side. The first needs a client who is doing one step at a time who can also skip patches if it needs (APT supports this for a long time now). The later is implemented by this commit, but depends on the server NOT merging the patches and the patches being in a strict order in which no patch is skipped. This is traditionally the case for dak, but other repository creators support merging – e.g. reprepro (which helpfully adds a flag indicating that the patches are merged). To support both or even mixes a client needs more information which isn't available for now. This POC uses the external diffindex-rred included in apt-file to do the heavy lifting of merging & applying all patches in one pass, hence to test this feature apt-file needs to be installed.
2013-12-13query an empty pkgAcqIndexDiffs if index is up-to-dateDavid Kalnischkies
The previous code already did this, this is just being a hell of a lot more obvious, so that it isn't that easy to break in the future. Git-Dch: Ignore
2013-10-09Merge remote-tracking branch 'mvo/feature/limit-default-pdiffs' into debian/sidMichael Vogt
2013-10-03use pkgAcqArchive in 'download' for proper errorsDavid Kalnischkies
With a bit of trickery we can reuse the usual infrastructure we have in place to acquire deb files for the 'download' operation as well, which gains us authentification check & display, error messages, correct filenames and "downloads" from the root-owned archives.
2013-10-03refactor onError relabeling of DestFile as '.FAILED'David Kalnischkies
This helps ensure three things: - each error is reported via ReportMirrorFailure - if DestFile doesn't exist, do not attempt rename - renames happen for every error The last one wasn't the case for Size mismatches, which isn't nice, but not a exploitable problem per-se as the file isn't picked up and remains in partial/ where the following download-try will at most take it for a partial request which fails the hashsum verification later on Git-Dch: Ignore
2013-09-26pkg from only trusted sources keeps being trustedDavid Kalnischkies
--allow-unauthenticated switches the download to a pre-0.6 system in which a package can come from any source, rather than that trusted packages can only come from trusted sources. To allow this the flag used to set all packages as untrusted, which is a bit much, so we check now if the package can be acquired via an untrusted source and only if this is the case set it as untrusted. As APT nowadays supports setting sources as trusted via a flag in the sources.list this mode shouldn't be used that much anymore though. [Note that this is not the patch from the BTS] Closes: 617690
2013-08-26set Acquire::PDiffs::FileLimit to 20 to avoid needless huge fetchesMichael Vogt
2013-08-22Merge remote-tracking branch 'mvo/bugfix/coverity' into debian/sidMichael Vogt
Conflicts: apt-pkg/tagfile.h
2013-08-12fix: --print-uris removes authenticationDavid Kalnischkies
The constructors of our (clear)sign-acquire-items move a pre-existent file for error-recovery away, which gets restored or discarded later as the acquire progresses, but --print-uris never really starts the acquire process, so the files aren't restored (as they should). To fix this both get a destructor which checks for signs of acquire doing anything and if it hasn't the file is restored. Note that these virtual destructors theoretically break the API, but only with classes extending the sign-acquire-items and nobody does this, as it would be insane for library users to fiddle with Acquire internals – and these classes are internals. Closes: 719263
2013-08-12some more coverity fixesMichael Vogt
2013-07-25pick up Translation-* even if only compressed availableDavid Kalnischkies
On CD-ROMs Translation-* files are only in compressed form included in the Release file. This used to work while we had no record of Translation-* files in the Release file at all as APT would have just guessed the (compressed) filename and accepted it (unchecked), but now that it checks for the presents of entries and if it finds records it expects the uncompressed to be verifiable. This commit relaxes this requirement again to fix the regression. We are still secure "enough" as we can validate the compressed file we have downloaded, so we don't loose anything by not requiring a hashsum for the uncompressed files to double-check them. Closes: 717665
2013-06-20do not redownload unchanged InRelease filesDavid Kalnischkies
Before we download the 'new' InRelease file the old file will be moved out of the way with the name 'foobar_InRelease.reverify', so if no partial file for the 'new' file exists take the modification time from this reverify file, so that if we get an IMS hit for the InRelease file we can move back the reverify file as new file rather than downloading the 'new' file even though we already have it. We do the same for Release files and this happened to work until the reverify renaming was corrected for InRelease files.
2013-04-10Fix English spelling error in a message ('A error'). Unfuzzybubulle@debian.org
translations. Closes: #705087
2013-04-08merged bundle from davidMichael Vogt
2013-04-02merged lp:~mvo/apt/fix-inrelease5Michael Vogt
2013-03-15* apt-pkg/acquire-item.cc:David Kalnischkies
- keep the last good InRelease file around just as we do it with Release.gpg in case the new one we download isn't good for us
2013-03-10various simple changes to fix cppcheck warningsDavid Kalnischkies
2013-01-14ensure sha512 is really used when available (thanks to Tyler Hicks )Michael Vogt
2012-03-06add Debug::pkgAcqArchive::NoQueue to disable package downloadingDavid Kalnischkies
2012-03-04* apt-pkg/acquire-item.cc:David Kalnischkies
- remove 'old' InRelease file if we can't get a new one before proceeding with Release.gpg to avoid the false impression of a still trusted repository by a (still present) old InRelease file. Thanks to Simon Ruderich for reporting this issue! (CVE-2012-0214) Effected are all versions >= 0.8.11 Possible attack summary: - Attacker needs to find a user which has run at least one successful 'apt-get update' against an archive providing InRelease files. - Create a Packages file with his preferred content. - Attacker then prevents the download of InRelease, Release and Release.gpg (alternatively he creates a valid Release file and sends this, the other two files need to be missing either way). - User updates against this, getting the modified Packages file without any indication of being unsigned (beside the "Ign InRelease" and "Ign Release.gpg" in the output of 'apt-get update'). => deb files from this source are considered 'trusted' (and therefore the user isn't asked for an additional confirmation before install)
2012-02-18use pdiff for Translation-* files if available (Closes: #657902)David Kalnischkies
Beware: pdiffs for Translation-* are only acquired if their availability is advertised in the Release file.
2012-02-18* apt-pkg/acquire-item.cc:David Kalnischkies
- drop support for i18n/Index file (introduced in 0.8.11) and use the Release file instead to get the Translations (Closes: #649314) * ftparchive/writer.cc: - add 'Translation-*' to the default patterns i18n/Index was never used outside debian - and even here it isn't used consistently as only 'main' has such a file. As the Release file now includes the Translation-* files we therefore drop support for i18n/Index. A version supporting it was never part of a debian release and still supporting it would mean that we get 99% of the time a 404 as response to the request anyway and confuse archive maintainers who want to provide all files APT tries to acquire.
2011-12-17try to avoid direct usage of .Fd() if possible and do read()s and coDavid Kalnischkies
on the FileFd instead
2011-09-19use forward declaration in headers if possible instead of includesDavid Kalnischkies
2011-09-13merge with debian/sidDavid Kalnischkies
2011-09-13merge with debian/experimentalDavid Kalnischkies
2011-09-13reorder includes: add <config.h> if needed and include it at firstDavid Kalnischkies