summaryrefslogtreecommitdiff
path: root/methods/http.h
AgeCommit message (Collapse)Author
2016-12-31separating state variables regarding server/requestDavid Kalnischkies
Having a Reset(bool) method to partially reset certain variables like the download size always were strange, so this commit splits the ServerState into an additional RequestState living on the stack for as long as we deal with this request causing an automatic "reset". There is much to do still to make this code look better, but this is a good first step which compiles cleanly and passes all tests, so keeping it as history might be beneficial and due to avoiding explicit memory allocations it ends up fixing a small memory leak in https, too. Closes: #440057
2016-08-16reorganize server-states resetting in http/httpsDavid Kalnischkies
We keep various information bits about the server around, some only effecting the currently handled file (like sizes) while others should be persistent (like pipeline detections). http used to reset all file-related manually, which is a bit silly if we already have a Reset() method – which does reset all through –, so extending it with a parameter for reuse and calling it from https too (as this was previously resetting by just creating a new state struct – it uses no value of the persistent state-keeping yet as it supports no pipelining). Gbp-Dch: Ignore
2016-08-10implement socks5h proxy support for http methodDavid Kalnischkies
Socks support is a requested feature in sofar that the internet is actually believing Acquire::socks::Proxy would exist. It doesn't and this commit isn't adding it as that isn't how our configuration works, but it allows Acquire::http::Proxy="socks5h://…". The HTTPS method was changed already to support socks proxies (all versions) via curl. This commit implements only SOCKS5 (RFC1928) with no auth or pass&user auth (RFC1929), but not GSSAPI which is required by the RFC. The 'h' in the protocol name further indicates that DNS resolution is delegated to the socks proxy rather than performed locally. The implementation works and was tested with Tor as socks proxy for which implementing socks5h only can actually be considered a feature. Closes: 744934
2016-08-10implement generic config fallback for methodsDavid Kalnischkies
The https method implemented for a long while now a hardcoded fallback to the same options in http, which, while it works, is rather inflexible if we want to allow the methods to use another name to change their behavior slightly, like apt-transport-tor does to https – most of the diff being s#https#tor#g which then fails to do the full circle fallthrough tor -> https -> http for https sources. With this config infrastructure this could be implemented now.
2016-08-10use the same redirection handling for http and httpsDavid Kalnischkies
cURL which backs our https implementation can handle redirects on its own, but by dealing with them on our own we gain finer control over which redirections will be performed (we don't like https → http) and by whom so that redirections to other hosts correctly spawn a new https method dealing with these instead of letting the current one deal with it.
2016-08-10detect redirection loops in acquire instead of workersDavid Kalnischkies
Having the detection handled in specific (http) workers means that a redirection loop over different hostnames isn't detected. Its also not a good idea have this implement in each method independently even if it would work
2016-07-05avoid 416 response teardown binding to null pointerDavid Kalnischkies
methods/http.cc:640:13: runtime error: reference binding to null pointer of type 'struct FileFd' This reference is never used in the cases it has a nullptr, so the practical difference is non-existent, but its a bug still. Reported-By: gcc -fsanitize=undefined
2016-05-28use std::locale::global instead of setlocaleDavid Kalnischkies
We use a wild mixture of C and C++ ways of generating output, so having a consistent world-view in both styles sounds like a good idea and should help in preventing regressions.
2015-11-05apply various suggestions made by cppcheckDavid Kalnischkies
Reported-By: cppcheck Git-Dch: Ignore
2015-11-05allow acquire method specific options via Binary scopeDavid Kalnischkies
Allows users who know what they are getting themselves into with this trick to e.g. disable privilege dropping for e.g. file:// until they can fix up the permissions on those repositories. It helps also the test framework and people with a similar setup (= me) to run in less modified environments.
2015-09-14fix two memory leaks reported by gccDavid Kalnischkies
Reported-By: gcc -fsanitize=address -fno-sanitize=vptr Git-Dch: Ignore
2015-08-10add c++11 override marker to overridden methodsDavid Kalnischkies
C++11 adds the 'override' specifier to mark that a method is overriding a base class method and error out if not. We hide it in the APT_OVERRIDE macro to ensure that we keep compiling in pre-c++11 standards. Reported-By: clang-modernize -add-override -override-macros Git-Dch: Ignore
2015-04-19calculate hashes while downloading in httpsDavid Kalnischkies
We do this in HTTP already to give the CPU some exercise while the disk is heavily spinning (or flashing?) to store the data avoiding the need to reread the entire file again later on to calculate the hashes – which happens outside of the eyes of progress reporting, so you might ended up with a bunch of https workers 'stuck' at 100% while they were busy calculating hashes. This is a bummer for everyone using apt as a connection speedtest as the https method works slower now (not really, it just isn't reporting done too early anymore).
2015-04-19calculate only expected hashes in methodsDavid Kalnischkies
Methods get told which hashes are expected by the acquire system, which means we can use this list to restrict what we calculate in the methods as any extra we are calculating is wasted effort as we can't compare it with anything anyway. Adding support for a new hash algorithm is therefore 'free' now and if a algorithm is no longer provided in a repository for a file, we automatically stop calculating it. In practice this results in a speed-up in Debian as we don't have SHA512 here (so far), so we practically stop calculating it.
2014-10-06make http size check workMichael Vogt
2014-09-02Make Proxy-Auto-Detect check for each hostMichael Vogt
When doing Acquire::http{,s}::Proxy-Auto-Detect, run the auto-detect command for each host instead of only once. This should make using "proxy" from libproxy-tools feasible which can then be used for PAC style or other proxy configurations. Closes: #759264
2014-08-26Pass ExpectedSize to tthe backend methodMichael Vogt
This ensures that we can stop downloading if the server send too much data by accident (or by a malicious attempt)
2014-03-13cleanup headers and especially #includes everywhereDavid Kalnischkies
Beside being a bit cleaner it hopefully also resolves oddball problems I have with high levels of parallel jobs. Git-Dch: Ignore Reported-By: iwyu (include-what-you-use)
2014-02-22Fix typos in documentation (codespell)Michael Vogt
2013-10-01handle complete responses to https range requestsDavid Kalnischkies
Servers might respond with a complete file either because they don't support Ranges at all or the If-Range condition isn't statisfied, so we have to parse the headers curl gets ourself to seek or truncate the file we have so far. This also finially adds the testcase testing a bunch of partial situations for both, http and https - which is now all green. Closes: 617643, 667699 LP: 1157943
2013-10-01refactor http client implementationDavid Kalnischkies
No effective behavior change, just shuffling big junks of code between methods and classes to split them into those strongly related to our client implementation and those implementing HTTP. The idea is to get HTTPS to a point in which most of the implementation can be shared even though the client implementations itself is completely different. This isn't anywhere near yet though, but it should beenough to reuse at least a few lines from http in https now. Git-Dch: Ignore
2013-04-10Fix English spelling error in a message ('A error'). Unfuzzybubulle@debian.org
translations. Closes: #705087
2012-03-04fix a bunch of cppcheck "(warning) Member variable '<#>' is notDavid Kalnischkies
initialized in the constructor." messages (no functional change)
2012-02-11remove the arbitrary MAXLEN limit for response lines (Closes: #658346)David Kalnischkies
2011-09-19use forward declaration in headers if possible instead of includesDavid Kalnischkies
2011-09-19do not pollute namespace in the headers with using (Closes: #500198)David Kalnischkies
2011-09-13merge with debian/experimentalDavid Kalnischkies
2011-09-13Support large files in the complete toolset. Indexes of thisDavid Kalnischkies
size are pretty unlikely for now, but we need it for deb packages which could become bigger than 4GB now (LP: #815895)
2011-08-11follow the recommendation of cppcheck to make some method methods (scnr)David Kalnischkies
const and initial mostly Debug member values in the constructors
2011-05-17ServerState is a struct and not a class (clang mismatch type warning)David Kalnischkies
2010-06-09fix compiler warning in the new mirror codeDavid Kalnischkies
2010-06-09* merge the remaining Ubuntu change:Michael Vogt
- on gpg verification failure warn and restore the last known good state - on failure display the IP of the server (useful for servers that use round robin DNS) - support Original-Maintainer in RewritePackageOrder - enable cdrom autodetection via libudev by default - show messsage about Vcs in use when apt-get source is run for packages maintained in a Vcs - better support transitional packages with mark auto-installed. when the transitional package is in "oldlibs" the new package is not marked auto installed (same is true for section metapackages) - provide new "deb mirror://archive.foo/mirrors.list sid main" method expects a list of mirrors (generated on the server e.g. via geoip) and will use that, including cycle on failure - write apport crash file on package failure (disabled by default on debian until apport is available) - support mirror failure reporting (disabled by default on debian)
2010-05-07* methods/http.cc: Michael Vogt
- code cleanup, add (some) doxygen strings
2010-02-17merged from the mvo branchMichael Vogt
2010-02-05add Acquire::http::ProxyAutoDetect configuration that Michael Vogt
can be used to call a external helper to figure out the proxy configuration and return it to apt via stdout
2009-02-02* [ABI break] merge support for http redirects, thanks toMichael Vogt
Jeff Licquia and Anthony Towns * [ABI break] use int for the package IDs (thanks to Steve Cotton) - Galician updated. Closes: #509151 - Catalan updated. Closes: #509375 - Vietnamese updated. Closes: #509422 - Traditional Chinese added. Closes: #510664 * COPYING: - Actualized. Removed obsolete Qt section, added GPLv2 clause. (Closes: #440049, #509337) * Clarify the --help for 'purge' (LP: #243948)
2009-01-30[ABI break] merge support for http redirects, thanks toMichael Vogt
Jeff Licquia and Anthony Towns
2007-07-12* cmdline/apt-get.cc:Michael Vogt
- fix in the task-install code regexp (thanks to Adam Conrad and Colin Watson). This should bring the livecd build back into shape * Fix compilation warnings: - apt-pkg/contrib/configuration.cc: wrong argument type; - apt-pkg/deb/dpkgpm.cc: wrong signess; - apt-pkg-acquire-item.cc: wrong signess and orderned initializers; - methods/https.cc: - type conversion; - unused variable; - changed SetupProxy() method to void;
2007-07-10Minor typos.Otavio Salvador
2007-06-14* apt-pkg/deb/dpkgpm.cc:Michael Vogt
- apport integration added, this means that a apport report is written on dpkg failures * merged from debian/unstable, remaining changes: - maintainer field changed * merged the debian/experimental changes back into the debian/sid branch * merged from Christian Perrier: * mr.po: New Marathi translation Closes: #416806 * zh_CN.po: Updated by Eric Pareja Closes: #416822 * tl.po: Updated by Eric Pareja Closes: #416638 * gl.po: Updated by Jacobo Tarrio * da.po: Updated by Claus Hindsgaul * fr.po: Remove a non-breakable space for usability * ru.po: Updated Russian translation. Closes: #405476 * *.po: Unfuzzy after upstream typo corrections * buildlib/archtable: - added support for sh3/sh4 (closes: #424870) - added support for m32r (closes: #394096) * buildlib/systemtable: - added support for lpia * configure.in: - check systemtable for architecture mapping too * fix error in AutocleanInterval, closes: #319339 (thanks to Israel G. Lugo for the patch) * add "purge" commandline argument, closes: #133421) (thanks to Julien Danjou for the patch) * add "purge" commandline argument, closes: #133421) (thanks to Julien Danjou for the patch) * fix FTBFS with gcc 4.3, closes: #417090 (thanks to Martin Michlmayr for the patch) * add --dsc-only option, thanks to K. Richard Pixley * Removed the more leftover #pragma interface/implementation closes: #306937 (thanks to Andreas Henriksson for the patch) * ABI library name change because its build against new glibc * implement SourceVer() in pkgRecords (thanks to Daniel Burrows for the patch!) * apt-pkg/algorithm.cc: - use clog for all debugging - only increase the score of installed applications if they are not obsolete - fix resolver bug on removal triggered by weak-dependencies with or-groups * methods/http.cc: - send apt version in User-Agent * apt-pkg/deb/debrecords.cc: - fix SHA1Hash() return value * apt-pkg/cdrom.cc: - only unmount if APT::CDROM::NoMount is false * methods/cdrom.cc: - only umount if it was mounted by the method before - if decompression of a index fails, delete the index * vi.po: Updated to 515t. Closes: #426976 * eu.po: Updated to 515t. Closes: #423766 * pt.po: 515t. Closes: #423111 * fr.po: Updated by Christian Perrier * Update all PO and the POT. Gives 513t2f for formerly complete translations * Package that contains tall the new features * Removed all #pragma interface/implementation * Branch that contains tall the new features: * translated package descriptions * task install support * automatic dependency removal (thanks to Daniel Burrows) * merged support for the new dpkg "Breaks" field (thanks to Ian Jackson) * handle network failures more gracefully on "update" * support for unattended-upgrades (via unattended-upgrades package) * added apt-transport-https method - ca.po: Updated to 514t - be.po: Updated to 514t - it.po: Updated to 514t - hu.po: Updated to 514t - zh_TW.po: Updated to 514t - ar.po: Updated to 293t221u. - ru.po: Updated to 514t. Closes: #392466 - nb.po: Updated to 514t. Closes: #392466 - pt.po: Updated to 514t. Closes: #393199 - fr.po: One spelling error corrected: s/accèder/accéder - km.po: Updated to 514t. - ko.po: Updated to 514t. - bg.po: Updated to 514t. - de.po: Updated to 514t. - en_GB.po: Updated to 514t. * debian/control: - depend on debian-archive-keyring to offer clean upgrade path (closes: #386800) * merged "install-recommends" branch (ABI break): - new "--install-recommends" - install new recommends on "upgrade" if --install-recommends is given - new "--fix-policy" option to install all packages with unmet important dependencies (usefull with --install-recommends to see what not-installed recommends are on the system) - fix of recommended packages display (only show CandidateVersion fix or-group handling) * merged "install-task" branch (use with "apt-get install taskname^") * Applied patch from Daniel Schepler to make apt bin-NMU able. * apt-pkg/acquire-item.cc: - fix reversed logic of the "Acquire::PDiffs" option * apt-pkg/contrib/sha256.cc: - applied patch to fix unaligned access problem. Closes: #367417 (thanks to David Mosberger) * apt-pkg/contrib/sha256.{cc,h},hashes.{cc,h}: support for sha256 (thanks to Anthony Towns) * ftparchive/cachedb.{cc,h},writer.{cc,h}: optimizations (thanks to Anthony Towns) * apt pdiff support from experimental merged * apt-pkg/deb/dpkgpm.cc: wording fixes (thanks to Matt Zimmerman) - fix error in dpkg interaction (closes: #364513, thanks to Martin Dickopp)
2007-06-09* debian/control: increase standards versionMichael Vogt
* methods/http.cc: revert pipeline depth and move global Proxy to here * methods/http.h: remove Proxy from here
2006-11-29* prototype of mirror method addedMichael Vogt
2005-11-22* merged http download limit for apt (#146877)Michael Vogt
2004-09-20Corrections for object lifetimeArch Librarian
Author: jgg Date: 2002-04-18 05:09:38 GMT Corrections for object lifetime
2004-09-20G++3 fixes from RandolphArch Librarian
Author: jgg Date: 2001-05-27 23:53:55 GMT G++3 fixes from Randolph
2004-09-20More SHA-1 prepArch Librarian
Author: jgg Date: 2001-03-06 07:15:29 GMT More SHA-1 prep
2004-09-20Join with aliencodeArch Librarian
Author: jgg Date: 2001-02-20 07:03:16 GMT Join with aliencode
2004-09-20Take careful care of persistence modesArch Librarian
Author: jgg Date: 2000-05-28 04:33:59 GMT Take careful care of persistence modes
2004-09-20Reorderd error handlingArch Librarian
Author: jgg Date: 1999-12-09 03:45:56 GMT Reorderd error handling
2004-09-20http pipelining controlArch Librarian
Author: jgg Date: 1998-12-10 05:39:53 GMT http pipelining control