summaryrefslogtreecommitdiff
path: root/methods/http.cc
AgeCommit message (Collapse)Author
2015-03-16derive more of https from http methodDavid Kalnischkies
Bug #778375 uncovered that https wasn't properly integrated in the class family tree of http as it was supposed to be leading to a NULL pointer dereference. Fixing this 'properly' was deemed to much diff for practically no gain that late in the release, so commit 0c2dc43d4fe1d026650b5e2920a021557f9534a6 just fixed the synptom, while this commit here is fixing the cause plus adding a test.
2014-12-09dispose http(s) 416 error page as non-contentDavid Kalnischkies
Real webservers (like apache) actually send an error page with a 416 response, but our client didn't expect it leaving the page on the socket to be parsed as response for the next request (http) or as file content (https), which isn't what we want at all… Symptom is a "Bad header line" as html usually doesn't parse that well to an http-header. This manifests itself e.g. if we have a complete file (or larger) in partial/ which isn't discarded by If-Range as the server doesn't support it (or it is just newer, think: mirror rotation). It is a sort-of regression of 78c72d0ce22e00b194251445aae306df357d5c1a, which removed the filesize - 1 trick, but this had its own problems… To properly test this our webserver gains the ability to reply with transfer-encoding: chunked as most real webservers will use it to send the dynamically generated error pages. Closes: 768797
2014-10-13Fix backward compatiblity of the new pkgAcquireMethod::DropPrivsOrDie()Michael Vogt
Do not drop privileges in the methods when using a older version of libapt that does not support the chown magic in partial/ yet. To do this DropPrivileges() now will ignore a empty Apt::Sandbox::User. Cleanup all hardcoded _apt along the way.
2014-10-07Send "Fail-Reason: MaximumSizeExceeded" from the methodMichael Vogt
Communicate the fail reason from the methods to the parent and Rename() failed files.
2014-10-07make expected-size a maximum-size check as this is what we want at this pointMichael Vogt
2014-10-06make http size check workMichael Vogt
2014-09-02Make Proxy-Auto-Detect check for each hostMichael Vogt
When doing Acquire::http{,s}::Proxy-Auto-Detect, run the auto-detect command for each host instead of only once. This should make using "proxy" from libproxy-tools feasible which can then be used for PAC style or other proxy configurations. Closes: #759264
2014-08-26Pass ExpectedSize to tthe backend methodMichael Vogt
This ensures that we can stop downloading if the server send too much data by accident (or by a malicious attempt)
2014-06-24methods/http.cc: use Req.str() in debug outputMichael Vogt
2014-04-26build http request in a stringstreamDavid Kalnischkies
beside reducing code a bit, it avoids oddball problems while building the string and doesn't trigger static analyse warnings.
2014-03-13follow method attribute suggestions by gccDavid Kalnischkies
Git-Dch: Ignore Reported-By: gcc -Wsuggest-attribute={pure,const,noreturn}
2014-03-13cleanup headers and especially #includes everywhereDavid Kalnischkies
Beside being a bit cleaner it hopefully also resolves oddball problems I have with high levels of parallel jobs. Git-Dch: Ignore Reported-By: iwyu (include-what-you-use)
2014-03-13StartPos is always positive for http/httpsDavid Kalnischkies
server.cc: In member function ‘bool ServerState::HeaderLine(std::string)’: server.cc:198:72: warning: format ‘%llu’ expects argument of type ‘long long unsigned int*’, but argument 3 has type ‘long long int*’ [-Wformat=] else if (sscanf(Val.c_str(),"bytes %llu-%*u/%llu",&StartPos,&Size) != 2) Git-Dch: Ignore Reported-By: gcc -Wpedantic
2014-03-13warning: extra ‘;’ [-Wpedantic]David Kalnischkies
Git-Dch: Ignore Reported-By: gcc -Wpedantic
2014-02-22Fix typos in documentation (codespell)Michael Vogt
2014-01-30fix various style/performance warnings in rredDavid Kalnischkies
Reported-By: cppcheck Git-Dch: Ignore
2014-01-16correct some style/performance/warnings from cppcheckDavid Kalnischkies
The most "visible" change is from utime to utimensat/futimens as the first one isn't part of POSIX anymore. Reported-By: cppcheck Git-Dch: Ignore
2013-10-22add Acquire::http::Proxy-Auto-Detect to the apt.conf.5 manpage (closes: 726597)Michael Vogt
2013-10-01handle complete responses to https range requestsDavid Kalnischkies
Servers might respond with a complete file either because they don't support Ranges at all or the If-Range condition isn't statisfied, so we have to parse the headers curl gets ourself to seek or truncate the file we have so far. This also finially adds the testcase testing a bunch of partial situations for both, http and https - which is now all green. Closes: 617643, 667699 LP: 1157943
2013-10-01refactor http client implementationDavid Kalnischkies
No effective behavior change, just shuffling big junks of code between methods and classes to split them into those strongly related to our client implementation and those implementing HTTP. The idea is to get HTTPS to a point in which most of the implementation can be shared even though the client implementations itself is completely different. This isn't anywhere near yet though, but it should beenough to reuse at least a few lines from http in https now. Git-Dch: Ignore
2013-10-01replace "filesize - 1" trick in http with proper 416 handlingDavid Kalnischkies
Our http client requests the "filesize - 1" for the small edgecase of handling a file which was completely downloaded, but not yet moved to the correct place as we get 416 errors in that case, but as we can handle 416 returns now we just special-case the situation of requesting the exact filesize and handle it as a 200 without content instead.
2013-10-01retry without partial data after a 416 responseDavid Kalnischkies
If we get a 416 from the server it means the Range we asked for is above the real filesize of the file on the server. Mostly this happens if the server isn't supporting If-Range, but regardless of how we end up with the partial data, the data is invalid so we discard it and retry with a fresh plate and hope for the best. Old behavior was to consider 416 an error and retry with a different compression until we ran out of compression and requested the uncompressed file (which doesn't exist on most mirrors) with an accept line which server answered with "406 Not Acceptable". Closes: 710924
2013-07-28Merge remote-tracking branch 'mvo/bugfix/coverity' into debian/sidMichael Vogt
2013-07-26request absolute URIs from proxies again (0.9.9.3 regession)David Kalnischkies
Commit 2b9c9b7f28b18f6ae3e422020e8934872b06c9f3 not only removes keep-alive, but also changes the request URI send to proxies which are required to be absolute URIs rather than the usual absolute paths. Closes: 717891
2013-07-25fix off-by-one error in HttpMethod::​AutoDetectProxy()Michael Vogt
2013-07-14Do not send a connection: keep-alive, at allRaphael Geissert
2013-05-08merged patch from Daniel Hartwig to fix URI and proxy releated issuesMichael Vogt
2013-01-24quote plus in filenames to work around a bug in the S3 serverMichael Vogt
(LP: #1003633)
2013-01-11* methods/http.cc:Michael Vogt
- quote spaces in filenames to ensure as the http method is also (potentially) used for non deb,dsc content that may contain spaces, thanks to Daniel Hartwig and Thomas Bushnell (LP: #1086997)
2012-05-17add spaces around PACKAGE_VERSION to fix FTBFS with -std=c++11David Kalnischkies
2012-05-15* methods/http.cc:David Kalnischkies
- after many years of pointless discussions disable http/1.1 pipelining by default as many webservers and proxies seem to be unable to conform to specification must's (rfc2616 section 8.1.2.2) (LP: #996151)
2012-05-14* apt-pkg/acquire*.cc:Raphael Geissert
- handle redirections in the worker with the right method instead of in the method the redirection occured in (Closes: #668111) * methods/http.cc: - forbid redirects to change protocol
2012-04-17newer gcc versions seems to have no problem with that, but while workingDavid Kalnischkies
with g++-4.1 it complains about this so lets be extra clear
2012-03-22the previously used VERSION didn't work everywhere so we are switchingDavid Kalnischkies
to the more standard PACKAGE_VERSION and make it work in every file
2012-03-05fix "(performance) Possible inefficient checking for 'R' emptiness."David Kalnischkies
2012-03-05fix "(style) Checking if unsigned variable 'Minor' is less than zero."David Kalnischkies
2012-03-05ensure that (s)scanf doesn't parse a too long Code now that a previousDavid Kalnischkies
commit lifted the Line-length limit
2012-02-11remove the arbitrary MAXLEN limit for response lines (Closes: #658346)David Kalnischkies
2012-01-30* methods/http{s,}.cc:David Kalnischkies
- if a file without an extension is requested send an 'Accept: text/*' header to avoid that the server chooses unsupported compressed files in a content-negotation attempt (Closes: #657560)
2011-12-17try to avoid direct usage of .Fd() if possible and do read()s and coDavid Kalnischkies
on the FileFd instead
2011-09-19use forward declaration in headers if possible instead of includesDavid Kalnischkies
2011-09-13merge with debian/experimentalDavid Kalnischkies
2011-09-13Support large files in the complete toolset. Indexes of thisDavid Kalnischkies
size are pretty unlikely for now, but we need it for deb packages which could become bigger than 4GB now (LP: #815895)
2011-09-13reorder includes: add <config.h> if needed and include it at firstDavid Kalnischkies
2011-08-11cppcheck complains about some possible speed improvements which could beDavid Kalnischkies
done on the mirco-optimazation level, so lets fix them: (performance) Possible inefficient checking for emptiness. (performance) Prefer prefix ++/-- operators for non-primitive types.
2011-05-10dequote URL taken from Location in redirects as we will otherwiseDavid Kalnischkies
quote an already quoted string in the request later (Closes: #602412)
2011-04-29Location header in redirects should be absolute URI, but someDavid Kalnischkies
servers just send an absolute path so still deal with it properly
2011-04-27* methods/http.cc:David Kalnischkies
- add config option to ignore a closed stdin to be able to easily use the method as a simple standalone downloader
2010-11-22* methods/http.cc:Michael Vogt
- do not hang if Acquire::http::ProxyAutoDetect can not be executed or returns no data (LP: #654393)
2010-08-21* methods/http.cc:David Kalnischkies
- some http servers violate HTTP1.1 by not issuing a Reason-Phrase (or at least a space after the code) especially for 200, but lets be nice and ignore it as we don't need the reason in general