summaryrefslogtreecommitdiff
path: root/apt-pkg/acquire-item.cc
AgeCommit message (Collapse)Author
2014-09-23fix testsMichael Vogt
2014-09-23Merge branch 'debian/sid' into debian/experimentalMichael Vogt
Conflicts: apt-pkg/acquire-item.cc apt-pkg/acquire-item.h apt-pkg/cachefilter.h configure.ac debian/changelog
2014-09-21Fix regression for cdrom: sources from latest security updateMichael Vogt
Skip a reverify for cdrom: sources. The reverify step is actually harmful here because the apt-cdrom add code uses the indexcopy.cc which will "normalize" the Packages file from the cdrom when it writes it to the local disk. This leads to changing the "MD5sum" field (notice the lower case "s") on the cdrom Packages file to a "MD5Sum" field on the local file in /var/lib/apt/lists. Which of course alters the hash and makes apt fail to reverify the file.
2014-09-17improve test for commit daff4aMichael Vogt
2014-09-17Fix regression for file:/// uris from CVE-2014-0487Michael Vogt
Do not run ReverifyAfterIMS() for local file URIs as this will causes apt to mess around in the file:/// uri space. This is wrong in itself, but it will also cause a incorrect verification failure when the archive and the lists directory are on different partitions as rename().
2014-09-16SECURITY UPDATE for CVE-2014-{0488,0487,0489}Michael Vogt
incorrect invalidating of unauthenticated data (CVE-2014-0488) incorect verification of 304 reply (CVE-2014-0487) incorrect verification of Acquire::Gzip indexes (CVE-2014-0489)
2014-09-05Merge branch 'debian/sid' into debian/experimentalMichael Vogt
Conflicts: apt-pkg/acquire-item.cc configure.ac debian/changelog doc/apt-verbatim.ent doc/po/apt-doc.pot doc/po/de.po doc/po/es.po doc/po/fr.po doc/po/it.po doc/po/ja.po doc/po/pt.po po/ar.po po/ast.po po/bg.po po/bs.po po/ca.po po/cs.po po/cy.po po/da.po po/de.po po/dz.po po/el.po po/es.po po/eu.po po/fi.po po/fr.po po/gl.po po/hu.po po/it.po po/ja.po po/km.po po/ko.po po/ku.po po/lt.po po/mr.po po/nb.po po/ne.po po/nl.po po/nn.po po/pl.po po/pt.po po/pt_BR.po po/ro.po po/ru.po po/sk.po po/sl.po po/sv.po po/th.po po/tl.po po/tr.po po/uk.po po/vi.po po/zh_CN.po po/zh_TW.po test/integration/test-ubuntu-bug-346386-apt-get-update-paywall
2014-07-17apt-pkg/acquire-item.cc: make pkgAcqDiffIndex more uniformMichael Vogt
2014-07-08Merge branch 'debian/sid' into debian/experimentalMichael Vogt
Conflicts: apt-pkg/deb/deblistparser.cc doc/po/apt-doc.pot doc/po/de.po doc/po/es.po doc/po/fr.po doc/po/it.po doc/po/ja.po doc/po/pl.po doc/po/pt.po doc/po/pt_BR.po po/da.po po/mr.po po/vi.po
2014-07-03Try not to parse invalid translation files (LP: #756317)Michael Vogt
2014-06-18Merge remote-tracking branch 'mvo/feature/update-by-hash' into ↵Michael Vogt
debian/experimental
2014-05-29Fix more warnings from clangMichael Vogt
Reported-By: clang++ -Werror Conflicts: apt-pkg/acquire-item.cc apt-pkg/acquire-item.h apt-pkg/deb/debmetaindex.h
2014-05-22move ByHash into its own functionMichael Vogt
2014-05-22Add APT::Acquire::$(host)::By-Hash=1 knob, add Acquire-By-Hash to Release fileMichael Vogt
The by-hash can be configured on a per-hostname basis and a Release file can indicate that it has by-hash support via a new flag. The location of the hash now matches the AptByHash spec
2014-05-22add TODOMichael Vogt
2014-05-22Implement simple by-hash for apt updateMichael Vogt
This implements a apt update schema that get the indexfiles by the hash instead of the name. The rational is that updates to the archive servers/mirrors are not atomic so the client may have the previous version of the Release file when the server updates to a new Release file and new Packages/Sources/Translations indexes. By keeping the files around by their hash we can still get the previous indexfile without a hashsum mismatch. Enable with APT::Acquire::By-Hash=1
2014-05-14Merge remote-tracking branch 'donkult/debian/experimental' into ↵Michael Vogt
debian/experimental Conflicts: apt-pkg/acquire-item.cc apt-pkg/acquire-item.h
2014-05-14Add new pkgAcqBaseIndex as base class for ↵Michael Vogt
pkgAcq{DiffIndex,IndexMerge,pkgAcqBaseIndex, pkgAcqIndex}
2014-05-14Pass struct IndexTarget/indexRecords to pkgAcqIndex{,Merge}DiffsMichael Vogt
If one of the pkgAcqIndex{,Merge}Diffs fails, they will run pkgAcqIndex() which needs the IndexTarget/indexRecords data. So we pass it along.
2014-05-09promote ExpectedHashes to a pkgAcquire::Item memberDavid Kalnischkies
Beside being another big API break with hopefully zero fallout in reality it avoids having the same member and helper code in each and every subclass.
2014-05-09use HashStringList in the acquire systemDavid Kalnischkies
It is not very extensible to have the supported Hashes hardcoded everywhere and especially if it is part of virtual method names. It is also possible that a method does not support the 'best' hash (yet), so we might end up not being able to verify a file even though we have a common subset of supported hashes. And those are just two of the cases in which it is handy to have a more dynamic selection. The downside is that this is a MAJOR API break, but the HashStringList has a string constructor for compatibility, so with a bit of luck the few frontends playing with the acquire system directly are okay.
2014-05-08Merge remote-tracking branch 'mvo/bugfix/update-progress-reporting' into ↵Michael Vogt
debian/experimental
2014-05-08Merge remote-tracking branch 'upstream/debian/sid' into ↵Michael Vogt
bugfix/update-progress-reporting
2014-05-07Merge remote-tracking branch 'mvo/bugfix/update-progress-reporting' into ↵Michael Vogt
debian/experimental
2014-05-07Merge remote-tracking branch 'mvo/feature/apt-install-deb' into ↵Michael Vogt
debian/experimental
2014-05-07Merge branch 'debian/sid' into debian/experimentalMichael Vogt
Conflicts: apt-pkg/cachefilter.h apt-pkg/contrib/fileutl.cc apt-pkg/contrib/netrc.h apt-pkg/deb/debsrcrecords.cc apt-pkg/init.h apt-pkg/pkgcache.cc debian/apt.install.in debian/changelog
2014-04-28initialize Verify in second pkgAcqIndex constructorDavid Kalnischkies
gcc reports in testcase ./test-bug-596498-trusted-unsigned-repo: apt-pkg/acquire-item.cc:1059:7: runtime error: load of value 234, which is not a valid value for type 'bool' This happens as the bool Verify is initialized only in one of the two constructors of the pkgAcqIndex class. It isn't a problem through as the verification controlled by this flag is optional and used to fail early on garbage files (like network portal pages) instead of later on in the hashsum verification or while parsing (the then untrusted) file. Reported-By: gcc-4.9 -fsanitize=undefined
2014-04-28WIP local deb installMichael Vogt
2014-04-17pass Target/MetaIndexParser around into pkgAcqDiffIndex so that this can be ↵Michael Vogt
used to create a proper pkgAcqIndex() with size information
2014-04-16use correct compression extension when loading the size of the size into the ↵Michael Vogt
fetcher
2014-04-16add Debug::acquire::progress debug option and fixme for index file loading ↵Michael Vogt
with the correct extension
2014-04-16load the size from the metaindex into the fetcher to have even more accurate ↵Michael Vogt
progress information
2014-04-16make the TotalFiles more reliable in apt-get updateMichael Vogt
2014-03-13use the pretty fullname of a pkg as download desciptionDavid Kalnischkies
Otherwise the "WARNING: The following packages cannot be authenticated!" messages does not include the architecture of the package, so it would be slightly misinformative.
2014-03-13follow method attribute suggestions by gccDavid Kalnischkies
Git-Dch: Ignore Reported-By: gcc -Wsuggest-attribute={pure,const,noreturn}
2014-03-13cleanup headers and especially #includes everywhereDavid Kalnischkies
Beside being a bit cleaner it hopefully also resolves oddball problems I have with high levels of parallel jobs. Git-Dch: Ignore Reported-By: iwyu (include-what-you-use)
2014-03-13warning: unused parameter ‘foo’ [-Wunused-parameter]David Kalnischkies
Reported-By: gcc -Wunused-parameter Git-Dch: Ignore
2014-03-13warning: useless cast to type A [-Wuseless-cast]David Kalnischkies
Git-Dch: Ignore Reported-By: gcc -Wuseless-cast
2014-02-27Merge branch 'debian/sid' into debian/experimentalMichael Vogt
Conflicts: apt-private/private-list.cc configure.ac debian/apt.install.in debian/changelog
2014-02-27initial version of apt-helperMichael Vogt
2014-02-22Fix typos in documentation (codespell)Michael Vogt
2014-02-10always cleanup patchfiles at the end of rred callDavid Kalnischkies
With APT::Get::List-Cleanup disabled the ed-style patch files are lingering in the lists/ directory otherwise. That was kinda okay in the old none-client-merge as the filename was always the same so it was constantly overridden, but now with different names for client-merge quiet a few could pill up on the system and are used by the next call as it picks them up based on the filename.
2014-01-15integrate Anthonys rred with POC for client-side mergeDavid Kalnischkies
Providing the benefits of both without the downsides :) (ABI breaks or external dependencies) For this Anthonys rred is equipped with: - magic-filename-pickup of patches rather than explicit messages - use of FileFd instead of FILE* to get on-the-fly uncompress of the gzip compressed pdiff patches The acquire code in turn stops checking for apt-file's helper as our own rred is now clever enough for our needs.
2014-01-15reenable unlimited pdiff files downloadDavid Kalnischkies
In 51fc6def77edfb1f429a48e5169519e9e05a759b we limited the amount of pdiff to be downloaded per index to 20. This was a compromise between not letting it go overboard (becoming even slower) and not using bandwidth needlessly. Now that with the POC the speed reason is gone it makes sense again to download as much files as we possible can via pdiff to save bandwidth (and possibly even time). It also avoids problems with the limit in cases we were we deal with a server merged archieve as this limit assumes a strict patch progression.
2014-01-15correct IndexDiff vs DiffIndex in Debug outputAnthony Towns
2013-12-13implement POC client-side merging of pdiffs via apt-fileDavid Kalnischkies
The idea of pdiffs is to avoid downloading the hole file by patching the existing index. This works very well, but becomes slow if a lot of patches needs to be applied to reconstruct an up-to-date index and in recent years more and more dinstall (or similar) runs are executed creating more and more pdiffs in the same amount of time, so pdiffs became less useful. The solution is simple: Reduce the amount of patches (which are very small) which need to be applied on top of the index we have available (which is usually pretty big). This can be done in two ways: Either merge the patches on the server-side so that the client has to download only one patch or the patches are all downloaded and merged on the client-side. The first needs a client who is doing one step at a time who can also skip patches if it needs (APT supports this for a long time now). The later is implemented by this commit, but depends on the server NOT merging the patches and the patches being in a strict order in which no patch is skipped. This is traditionally the case for dak, but other repository creators support merging – e.g. reprepro (which helpfully adds a flag indicating that the patches are merged). To support both or even mixes a client needs more information which isn't available for now. This POC uses the external diffindex-rred included in apt-file to do the heavy lifting of merging & applying all patches in one pass, hence to test this feature apt-file needs to be installed.
2013-12-13query an empty pkgAcqIndexDiffs if index is up-to-dateDavid Kalnischkies
The previous code already did this, this is just being a hell of a lot more obvious, so that it isn't that easy to break in the future. Git-Dch: Ignore
2013-10-09Merge remote-tracking branch 'mvo/feature/limit-default-pdiffs' into debian/sidMichael Vogt
2013-10-03use pkgAcqArchive in 'download' for proper errorsDavid Kalnischkies
With a bit of trickery we can reuse the usual infrastructure we have in place to acquire deb files for the 'download' operation as well, which gains us authentification check & display, error messages, correct filenames and "downloads" from the root-owned archives.
2013-10-03refactor onError relabeling of DestFile as '.FAILED'David Kalnischkies
This helps ensure three things: - each error is reported via ReportMirrorFailure - if DestFile doesn't exist, do not attempt rename - renames happen for every error The last one wasn't the case for Size mismatches, which isn't nice, but not a exploitable problem per-se as the file isn't picked up and remains in partial/ where the following download-try will at most take it for a partial request which fails the hashsum verification later on Git-Dch: Ignore