summaryrefslogtreecommitdiff
path: root/methods
AgeCommit message (Collapse)Author
2014-10-07methods/https.cc: use File->Tell() here tooMichael Vogt
2014-10-06make http size check workMichael Vogt
2014-09-29Merge branch 'debian/sid' into debian/experimentalMichael Vogt
Conflicts: apt-pkg/acquire-item.cc
2014-09-27fix: %i in format string (no. 1) requires 'int' but the argument type isDavid Kalnischkies
'unsigned int' Git-Dch: Ignore Reported-By: cppcheck
2014-09-27correct the error messages to refer to apt-key instead of gpgvDavid Kalnischkies
Git-Dch: Ignore
2014-09-27fix: Member variable 'X' is not initialized in the constructor.David Kalnischkies
Reported-By: cppcheck Git-Dch: Ignore
2014-09-26Disable Mth.DropPrivsOrDie() in copy.cc for nowMichael Vogt
Dch-Ignore: true
2014-09-24releasing package apt version 1.1~exp3Michael Vogt
2014-09-24methods: Fail if we cannot drop privilegesJulian Andres Klode
2014-09-24Drop Privileges to "Debian-apt" in most acquire methodsMichael Vogt
Add a new "Debian-apt" user that owns the /var/lib/apt/lists and /var/cache/apt/archive directories. The methods http, https, ftp, gpgv, gzip switch to this user when they start. Thanks to Julian and "ioerror" and tors "switch_id()" code.
2014-09-23Merge branch 'debian/sid' into debian/experimentalMichael Vogt
Conflicts: apt-pkg/acquire-item.cc apt-pkg/acquire-item.h apt-pkg/cachefilter.h configure.ac debian/changelog
2014-09-21generalize Acquire::GzipIndexMichael Vogt
2014-09-19Fix regression when copy: is used for a relative pathMichael Vogt
When we do a ReverifyAfterIMS() we use the copy: method to verify the hashes again. If the user uses -o Dir=./something/relative this fails because we use the URI class in copy.cc that strips away the leading relative part. By not using URI this is fixed. Closes: #762160
2014-09-16SECURITY UPDATE for CVE-2014-{0488,0487,0489}Michael Vogt
incorrect invalidating of unauthenticated data (CVE-2014-0488) incorect verification of 304 reply (CVE-2014-0487) incorrect verification of Acquire::Gzip indexes (CVE-2014-0489)
2014-09-05Improve Debug::Acquire::http debug outputMichael Vogt
Prefix all answers with the URL that the answer is for. This helps when debugging and pipeline is enabled.
2014-09-05Merge branch 'debian/sid' into debian/experimentalMichael Vogt
Conflicts: apt-pkg/acquire-item.cc configure.ac debian/changelog doc/apt-verbatim.ent doc/po/apt-doc.pot doc/po/de.po doc/po/es.po doc/po/fr.po doc/po/it.po doc/po/ja.po doc/po/pt.po po/ar.po po/ast.po po/bg.po po/bs.po po/ca.po po/cs.po po/cy.po po/da.po po/de.po po/dz.po po/el.po po/es.po po/eu.po po/fi.po po/fr.po po/gl.po po/hu.po po/it.po po/ja.po po/km.po po/ko.po po/ku.po po/lt.po po/mr.po po/nb.po po/ne.po po/nl.po po/nn.po po/pl.po po/pt.po po/pt_BR.po po/ro.po po/ru.po po/sk.po po/sl.po po/sv.po po/th.po po/tl.po po/tr.po po/uk.po po/vi.po po/zh_CN.po po/zh_TW.po test/integration/test-ubuntu-bug-346386-apt-get-update-paywall
2014-09-02Make Proxy-Auto-Detect check for each hostMichael Vogt
When doing Acquire::http{,s}::Proxy-Auto-Detect, run the auto-detect command for each host instead of only once. This should make using "proxy" from libproxy-tools feasible which can then be used for PAC style or other proxy configurations. Closes: #759264
2014-08-26make https honor ExpectedSize as wellMichael Vogt
2014-08-26Pass ExpectedSize to tthe backend methodMichael Vogt
This ensures that we can stop downloading if the server send too much data by accident (or by a malicious attempt)
2014-07-08Merge branch 'debian/sid' into debian/experimentalMichael Vogt
Conflicts: apt-pkg/deb/deblistparser.cc doc/po/apt-doc.pot doc/po/de.po doc/po/es.po doc/po/fr.po doc/po/it.po doc/po/ja.po doc/po/pl.po doc/po/pt.po doc/po/pt_BR.po po/da.po po/mr.po po/vi.po
2014-06-24methods/http.cc: use Req.str() in debug outputMichael Vogt
2014-05-09reenable pipelining via hashsum reordering supportDavid Kalnischkies
Now that methods have the expected hashes available they can check if the response from the server is what they expected. Pipelining is one of those areas in which servers can mess up by not supporting it properly, which forced us to disable it for the time being. Now, we check if we got a response out of order, which we can not only use to disable pipelining automatically for the next requests, but we can fix it up just like the server responded in proper order for the current requests. To ensure that this little trick works pipelining is only attempt if we have hashsums for all the files in the chain which in theory reduces the use of pipelining usage even on the many servers which work properly, but in practice only the InRelease file (or similar such) will be requested without a hashsum – and as it is the only file requested in that stage it can't be pipelined even if we wanted to. Some minor annoyances remain: The display of the progress we have doesn't reflect this change, so it looks like the same package gets downloaded multiple times while others aren't at all. Further more, partial files are not supported in this recovery as the received data was appended to the wrong file, so the hashsum doesn't match. Both seem to be minor enough to reenable pipelining by default until further notice through to test if it really solves the problem. This therefore reverts commit 8221431757c775ee875a061b184b5f6f2330f928.
2014-04-26enforce LFS for partial files in https range requestsDavid Kalnischkies
2014-04-26build http request in a stringstreamDavid Kalnischkies
beside reducing code a bit, it avoids oddball problems while building the string and doesn't trigger static analyse warnings.
2014-03-13add a config option to switch uncompress methods to compressDavid Kalnischkies
Not very useful in the normal operation of work, but handy for tests. Git-Dch: Ignore
2014-03-13follow method attribute suggestions by gccDavid Kalnischkies
Git-Dch: Ignore Reported-By: gcc -Wsuggest-attribute={pure,const,noreturn}
2014-03-13cleanup headers and especially #includes everywhereDavid Kalnischkies
Beside being a bit cleaner it hopefully also resolves oddball problems I have with high levels of parallel jobs. Git-Dch: Ignore Reported-By: iwyu (include-what-you-use)
2014-03-13warning: unused parameter ‘foo’ [-Wunused-parameter]David Kalnischkies
Reported-By: gcc -Wunused-parameter Git-Dch: Ignore
2014-03-13StartPos is always positive for http/httpsDavid Kalnischkies
server.cc: In member function ‘bool ServerState::HeaderLine(std::string)’: server.cc:198:72: warning: format ‘%llu’ expects argument of type ‘long long unsigned int*’, but argument 3 has type ‘long long int*’ [-Wformat=] else if (sscanf(Val.c_str(),"bytes %llu-%*u/%llu",&StartPos,&Size) != 2) Git-Dch: Ignore Reported-By: gcc -Wpedantic
2014-03-13warning: extra ‘;’ [-Wpedantic]David Kalnischkies
Git-Dch: Ignore Reported-By: gcc -Wpedantic
2014-03-13fix -Wformat= warnings about size_t != %lu on e.g. armelDavid Kalnischkies
Git-Dch: Ignore Reported-By: gcc
2014-03-13fix -Wmissing-field-initializers warningsDavid Kalnischkies
Reported-By: gcc Git-Dch: Ignore
2014-02-22Fix typos in documentation (codespell)Michael Vogt
2014-02-14add a testcase to check for forbidden https→http downgradesDavid Kalnischkies
Git-Dch: Ignore
2014-02-14disable https->http redirects in libcurlMichael Vogt
This change prevents changing the protocol from https to http.
2014-02-14allow http protocol to switch to httpsDavid Kalnischkies
switch protocols at random is a bad idea if e.g. http can switch to file, so we limit the possibilities to http to http and http to https. As very few people (less than 1% according to popcon) have https installed this likely changes nothing in terms of failure. The commit is adding a friendly hint which package needs to be installed though.
2014-02-14report https download start only if we really get itDavid Kalnischkies
Reporting it via progress means that e.g. a redirect will trigger it, too, so you get a Get & Hit while http only reports a Hit as it should be.
2014-02-11use utimes instead of utimensat/futimensDavid Kalnischkies
cppcheck complains about the obsolete utime as it was removed in POSIX1.2008 and recommends usage of utimensat/futimens instead as those are in POSIX and so commit 9ce3cfc9 switched to them. It is just that they aren't as portable as the standard suggests: At least our kFreeBSD and Hurd ports stumble over it at runtime. So to make both, the ports and cppcheck happy, we use utimes instead. Closes: 738567
2014-01-30fix various style/performance warnings in rredDavid Kalnischkies
Reported-By: cppcheck Git-Dch: Ignore
2014-01-26Merge remote-tracking branch 'donkult/debian/sid' into debian/sidMichael Vogt
Conflicts: apt-private/private-list.cc doc/po/de.po test/integration/framework
2014-01-21methods/rred: minor robustness improvementsAnthony Towns
Use retry_fwrite to better handle partial fwrite successes, and to keep the Hashes in sync with what's actually written.
2014-01-16correct some style/performance/warnings from cppcheckDavid Kalnischkies
The most "visible" change is from utime to utimensat/futimens as the first one isn't part of POSIX anymore. Reported-By: cppcheck Git-Dch: Ignore
2014-01-15integrate Anthonys rred with POC for client-side mergeDavid Kalnischkies
Providing the benefits of both without the downsides :) (ABI breaks or external dependencies) For this Anthonys rred is equipped with: - magic-filename-pickup of patches rather than explicit messages - use of FileFd instead of FILE* to get on-the-fly uncompress of the gzip compressed pdiff patches The acquire code in turn stops checking for apt-file's helper as our own rred is now clever enough for our needs.
2014-01-15reimplement rred to allow applying all the diffs in a single passAnthony Towns
Based on the idea presented in: https://lists.debian.org/deity/2009/08/msg00169.html and https://lists.debian.org/debian-devel/2014/01/msg00081.html It reads all patches one by one and merges them in-memory before applying the merged changes to the index. Beware: This commit by David Kalnischkies rips out the rred binary rewrite unchanged (expect minor format issue corrections) from the proposed changes, so this commit alone BREAKS pdiff completely. The integration into the acquire system as it was prepared in the previous POC will be done in the next commit to have proper 'blame'.
2013-12-13implement POC client-side merging of pdiffs via apt-fileDavid Kalnischkies
The idea of pdiffs is to avoid downloading the hole file by patching the existing index. This works very well, but becomes slow if a lot of patches needs to be applied to reconstruct an up-to-date index and in recent years more and more dinstall (or similar) runs are executed creating more and more pdiffs in the same amount of time, so pdiffs became less useful. The solution is simple: Reduce the amount of patches (which are very small) which need to be applied on top of the index we have available (which is usually pretty big). This can be done in two ways: Either merge the patches on the server-side so that the client has to download only one patch or the patches are all downloaded and merged on the client-side. The first needs a client who is doing one step at a time who can also skip patches if it needs (APT supports this for a long time now). The later is implemented by this commit, but depends on the server NOT merging the patches and the patches being in a strict order in which no patch is skipped. This is traditionally the case for dak, but other repository creators support merging – e.g. reprepro (which helpfully adds a flag indicating that the patches are merged). To support both or even mixes a client needs more information which isn't available for now. This POC uses the external diffindex-rred included in apt-file to do the heavy lifting of merging & applying all patches in one pass, hence to test this feature apt-file needs to be installed.
2013-10-22add Acquire::http::Proxy-Auto-Detect to the apt.conf.5 manpage (closes: 726597)Michael Vogt
2013-10-01handle complete responses to https range requestsDavid Kalnischkies
Servers might respond with a complete file either because they don't support Ranges at all or the If-Range condition isn't statisfied, so we have to parse the headers curl gets ourself to seek or truncate the file we have so far. This also finially adds the testcase testing a bunch of partial situations for both, http and https - which is now all green. Closes: 617643, 667699 LP: 1157943
2013-10-01fix partial (206 and 416) support in httpsDavid Kalnischkies
As lengthy discussed in lp:1157943 partial https support was utterly broken as a 206 response was handled as an (unhandled) error. This is the first part of fixing it by supporting a 206 response and starting to deal with 416.
2013-10-01refactor http client implementationDavid Kalnischkies
No effective behavior change, just shuffling big junks of code between methods and classes to split them into those strongly related to our client implementation and those implementing HTTP. The idea is to get HTTPS to a point in which most of the implementation can be shared even though the client implementations itself is completely different. This isn't anywhere near yet though, but it should beenough to reuse at least a few lines from http in https now. Git-Dch: Ignore
2013-10-01replace "filesize - 1" trick in http with proper 416 handlingDavid Kalnischkies
Our http client requests the "filesize - 1" for the small edgecase of handling a file which was completely downloaded, but not yet moved to the correct place as we get 416 errors in that case, but as we can handle 416 returns now we just special-case the situation of requesting the exact filesize and handle it as a 200 without content instead.