| Age | Commit message (Collapse) | Author |
|---|
|
We do not want to end up in a code path while reading content
from the server where we have local data left to write, which
can happen if a previous read included both headers and content.
Restructure Flush() to accept a new argument to allow incomplete
flushs (which do not match our limit), so that it can flush as
far as possible, and modify Go() and use that before and after
reading from the server.
|
|
Using the time of day for this is slightly wrong just like it is for
progress, just less visible.
|
|
The Stats method isn't called anywhere, was partly commented out before,
but we keep updating the time for it – lets avoid this pointless busywork.
Gbp-Dch: Ignore
|
|
Prompted-by: Jakub Wilk <jwilk@debian.org>
|
|
The Fail method for acquire methods has a boolean parameter indicating
the transient-nature of a reported error. The problem with this is that
Fail is called very late at a point where it is no longer easily
identifiable if an error is indeed transient or not, so some calls were
and some weren't and the acquire system would later mostly ignore the
transient flag and guess by using the FailReason instead.
Introducing a tri-state enum we can pass the information about fatal or
transient errors through the callstack to generate the correct fails.
|
|
Opening the file before we drop privileges in the methods allows us to
avoid chowning in the acquire main process which can apply to the wrong
file (imagine Binary scoped settings) and surprises users as their
permission setup is overridden.
There are no security benefits as the file is open, so an evil method
could as before read the contents of the file, but it isn't worse than
before and we avoid permission problems in this setup.
|
|
Proxying HTTPS traffic requires the proxy providing the
CONNECT method. This implements the client side of it,
although it is a bit hacky.
HTTP connect is a normal HTTP CONNECT request, followed
by a normal HTTP response, just that the body of the
response is the TCP stream of the target host.
We use a special wrapper in case there are data bytes
in the header packets - in that case, the bytes are
stored in a buffer and the buffer will be drained first,
afterwards the connection continues directly with the
TCP stream (with one more vcall).
Also: Do not send full URI to https destinations when proxying,
as we are directly interfacing with the destination data stream.
|
|
Use std::unique_ptr<MethodFd> everywhere we used an
integer-based file descriptor before. This allows us
to implement stuff like TLS support easily.
Gbp-Dch: ignore
|
|
This 'method' is the abstract base for http and https and should as such
be called out like this rather using an easily confused name.
Gbp-Dch: Ignore
|
|
Having a Reset(bool) method to partially reset certain variables like
the download size always were strange, so this commit splits the
ServerState into an additional RequestState living on the stack for as
long as we deal with this request causing an automatic "reset".
There is much to do still to make this code look better, but this is a
good first step which compiles cleanly and passes all tests, so keeping
it as history might be beneficial and due to avoiding explicit memory
allocations it ends up fixing a small memory leak in https, too.
Closes: #440057
|
|
We keep various information bits about the server around, some only
effecting the currently handled file (like sizes) while others
should be persistent (like pipeline detections). http used to reset all
file-related manually, which is a bit silly if we already have a Reset()
method – which does reset all through –, so extending it with a
parameter for reuse and calling it from https too (as this was
previously resetting by just creating a new state struct – it uses no
value of the persistent state-keeping yet as it supports no pipelining).
Gbp-Dch: Ignore
|
|
Socks support is a requested feature in sofar that the internet is
actually believing Acquire::socks::Proxy would exist. It doesn't and
this commit isn't adding it as that isn't how our configuration works,
but it allows Acquire::http::Proxy="socks5h://…". The HTTPS method was
changed already to support socks proxies (all versions) via curl. This
commit implements only SOCKS5 (RFC1928) with no auth or pass&user auth
(RFC1929), but not GSSAPI which is required by the RFC. The 'h' in the
protocol name further indicates that DNS resolution is delegated to the
socks proxy rather than performed locally.
The implementation works and was tested with Tor as socks proxy for
which implementing socks5h only can actually be considered a feature.
Closes: 744934
|
|
The https method implemented for a long while now a hardcoded fallback
to the same options in http, which, while it works, is rather inflexible
if we want to allow the methods to use another name to change their
behavior slightly, like apt-transport-tor does to https – most of the
diff being s#https#tor#g which then fails to do the full circle
fallthrough tor -> https -> http for https sources. With this config
infrastructure this could be implemented now.
|
|
cURL which backs our https implementation can handle redirects on its
own, but by dealing with them on our own we gain finer control over which
redirections will be performed (we don't like https → http) and by whom
so that redirections to other hosts correctly spawn a new https method
dealing with these instead of letting the current one deal with it.
|
|
Having the detection handled in specific (http) workers means that a
redirection loop over different hostnames isn't detected. Its also not a
good idea have this implement in each method independently even if it
would work
|
|
methods/http.cc:640:13: runtime error: reference binding to null pointer
of type 'struct FileFd'
This reference is never used in the cases it has a nullptr, so the
practical difference is non-existent, but its a bug still.
Reported-By: gcc -fsanitize=undefined
|
|
We use a wild mixture of C and C++ ways of generating output, so having
a consistent world-view in both styles sounds like a good idea and
should help in preventing regressions.
|
|
Reported-By: cppcheck
Git-Dch: Ignore
|
|
Allows users who know what they are getting themselves into with this
trick to e.g. disable privilege dropping for e.g. file:// until they can
fix up the permissions on those repositories. It helps also the test
framework and people with a similar setup (= me) to run in less modified
environments.
|
|
Reported-By: gcc -fsanitize=address -fno-sanitize=vptr
Git-Dch: Ignore
|
|
C++11 adds the 'override' specifier to mark that a method is overriding
a base class method and error out if not. We hide it in the APT_OVERRIDE
macro to ensure that we keep compiling in pre-c++11 standards.
Reported-By: clang-modernize -add-override -override-macros
Git-Dch: Ignore
|
|
We do this in HTTP already to give the CPU some exercise while the disk
is heavily spinning (or flashing?) to store the data avoiding the need
to reread the entire file again later on to calculate the hashes – which
happens outside of the eyes of progress reporting, so you might ended up
with a bunch of https workers 'stuck' at 100% while they were busy
calculating hashes.
This is a bummer for everyone using apt as a connection speedtest as the
https method works slower now (not really, it just isn't reporting done
too early anymore).
|
|
Methods get told which hashes are expected by the acquire system, which
means we can use this list to restrict what we calculate in the methods
as any extra we are calculating is wasted effort as we can't compare it
with anything anyway.
Adding support for a new hash algorithm is therefore 'free' now and if a
algorithm is no longer provided in a repository for a file, we
automatically stop calculating it.
In practice this results in a speed-up in Debian as we don't have SHA512
here (so far), so we practically stop calculating it.
|
|
|
|
When doing Acquire::http{,s}::Proxy-Auto-Detect, run the auto-detect
command for each host instead of only once. This should make using
"proxy" from libproxy-tools feasible which can then be used for PAC
style or other proxy configurations.
Closes: #759264
|
|
This ensures that we can stop downloading if the server send
too much data by accident (or by a malicious attempt)
|
|
Beside being a bit cleaner it hopefully also resolves oddball problems
I have with high levels of parallel jobs.
Git-Dch: Ignore
Reported-By: iwyu (include-what-you-use)
|
|
|
|
Servers might respond with a complete file either because they don't
support Ranges at all or the If-Range condition isn't statisfied, so we
have to parse the headers curl gets ourself to seek or truncate the file
we have so far.
This also finially adds the testcase testing a bunch of partial
situations for both, http and https - which is now all green.
Closes: 617643, 667699
LP: 1157943
|
|
No effective behavior change, just shuffling big junks of code between
methods and classes to split them into those strongly related to our
client implementation and those implementing HTTP.
The idea is to get HTTPS to a point in which most of the implementation
can be shared even though the client implementations itself is
completely different. This isn't anywhere near yet though, but it should
beenough to reuse at least a few lines from http in https now.
Git-Dch: Ignore
|
|
translations. Closes: #705087
|
|
initialized in the constructor." messages (no functional change)
|
|
|
|
|
|
|
|
|
|
size are pretty unlikely for now, but we need it for deb
packages which could become bigger than 4GB now (LP: #815895)
|
|
const and initial mostly Debug member values in the constructors
|
|
|
|
|
|
- on gpg verification failure warn and restore the last known
good state
- on failure display the IP of the server (useful for servers
that use round robin DNS)
- support Original-Maintainer in RewritePackageOrder
- enable cdrom autodetection via libudev by default
- show messsage about Vcs in use when apt-get source is run for
packages maintained in a Vcs
- better support transitional packages with mark auto-installed.
when the transitional package is in "oldlibs" the new package
is not marked auto installed (same is true for section
metapackages)
- provide new "deb mirror://archive.foo/mirrors.list sid main"
method expects a list of mirrors (generated on the server e.g.
via geoip) and will use that, including cycle on failure
- write apport crash file on package failure (disabled by default
on debian until apport is available)
- support mirror failure reporting (disabled by default on debian)
|
|
- code cleanup, add (some) doxygen strings
|
|
|
|
can be used to call a external helper to figure out the
proxy configuration and return it to apt via stdout
|
|
Jeff Licquia and Anthony Towns
* [ABI break] use int for the package IDs (thanks to Steve Cotton)
- Galician updated. Closes: #509151
- Catalan updated. Closes: #509375
- Vietnamese updated. Closes: #509422
- Traditional Chinese added. Closes: #510664
* COPYING:
- Actualized. Removed obsolete Qt section, added GPLv2 clause.
(Closes: #440049, #509337)
* Clarify the --help for 'purge' (LP: #243948)
|
|
Jeff Licquia and Anthony Towns
|
|
- fix in the task-install code regexp (thanks to Adam Conrad and
Colin Watson). This should bring the livecd build back into
shape
* Fix compilation warnings:
- apt-pkg/contrib/configuration.cc: wrong argument type;
- apt-pkg/deb/dpkgpm.cc: wrong signess;
- apt-pkg-acquire-item.cc: wrong signess and orderned initializers;
- methods/https.cc:
- type conversion;
- unused variable;
- changed SetupProxy() method to void;
|
|
|
|
- apport integration added, this means that a apport
report is written on dpkg failures
* merged from debian/unstable, remaining changes:
- maintainer field changed
* merged the debian/experimental changes back
into the debian/sid branch
* merged from Christian Perrier:
* mr.po: New Marathi translation Closes: #416806
* zh_CN.po: Updated by Eric Pareja Closes: #416822
* tl.po: Updated by Eric Pareja Closes: #416638
* gl.po: Updated by Jacobo Tarrio
* da.po: Updated by Claus Hindsgaul
* fr.po: Remove a non-breakable space for usability
* ru.po: Updated Russian translation. Closes: #405476
* *.po: Unfuzzy after upstream typo corrections
* buildlib/archtable:
- added support for sh3/sh4 (closes: #424870)
- added support for m32r (closes: #394096)
* buildlib/systemtable:
- added support for lpia
* configure.in:
- check systemtable for architecture mapping too
* fix error in AutocleanInterval, closes: #319339
(thanks to Israel G. Lugo for the patch)
* add "purge" commandline argument, closes: #133421)
(thanks to Julien Danjou for the patch)
* add "purge" commandline argument, closes: #133421)
(thanks to Julien Danjou for the patch)
* fix FTBFS with gcc 4.3, closes: #417090
(thanks to Martin Michlmayr for the patch)
* add --dsc-only option, thanks to K. Richard Pixley
* Removed the more leftover #pragma interface/implementation
closes: #306937 (thanks to Andreas Henriksson for the patch)
* ABI library name change because its build against
new glibc
* implement SourceVer() in pkgRecords
(thanks to Daniel Burrows for the patch!)
* apt-pkg/algorithm.cc:
- use clog for all debugging
- only increase the score of installed applications if they
are not obsolete
- fix resolver bug on removal triggered by weak-dependencies
with or-groups
* methods/http.cc:
- send apt version in User-Agent
* apt-pkg/deb/debrecords.cc:
- fix SHA1Hash() return value
* apt-pkg/cdrom.cc:
- only unmount if APT::CDROM::NoMount is false
* methods/cdrom.cc:
- only umount if it was mounted by the method before
- if decompression of a index fails, delete the index
* vi.po: Updated to 515t. Closes: #426976
* eu.po: Updated to 515t. Closes: #423766
* pt.po: 515t. Closes: #423111
* fr.po: Updated by Christian Perrier
* Update all PO and the POT. Gives 513t2f for formerly
complete translations
* Package that contains tall the new features
* Removed all #pragma interface/implementation
* Branch that contains tall the new features:
* translated package descriptions
* task install support
* automatic dependency removal (thanks to Daniel Burrows)
* merged support for the new dpkg "Breaks" field
(thanks to Ian Jackson)
* handle network failures more gracefully on "update"
* support for unattended-upgrades (via unattended-upgrades
package)
* added apt-transport-https method
- ca.po: Updated to 514t
- be.po: Updated to 514t
- it.po: Updated to 514t
- hu.po: Updated to 514t
- zh_TW.po: Updated to 514t
- ar.po: Updated to 293t221u.
- ru.po: Updated to 514t. Closes: #392466
- nb.po: Updated to 514t. Closes: #392466
- pt.po: Updated to 514t. Closes: #393199
- fr.po: One spelling error corrected: s/accèder/accéder
- km.po: Updated to 514t.
- ko.po: Updated to 514t.
- bg.po: Updated to 514t.
- de.po: Updated to 514t.
- en_GB.po: Updated to 514t.
* debian/control:
- depend on debian-archive-keyring to offer clean upgrade path
(closes: #386800)
* merged "install-recommends" branch (ABI break):
- new "--install-recommends"
- install new recommends on "upgrade" if --install-recommends is
given
- new "--fix-policy" option to install all packages with unmet
important dependencies (usefull with --install-recommends to
see what not-installed recommends are on the system)
- fix of recommended packages display (only show CandidateVersion
fix or-group handling)
* merged "install-task" branch (use with "apt-get install taskname^")
* Applied patch from Daniel Schepler to make apt bin-NMU able.
* apt-pkg/acquire-item.cc:
- fix reversed logic of the "Acquire::PDiffs" option
* apt-pkg/contrib/sha256.cc:
- applied patch to fix unaligned access problem. Closes: #367417
(thanks to David Mosberger)
* apt-pkg/contrib/sha256.{cc,h},hashes.{cc,h}: support for sha256
(thanks to Anthony Towns)
* ftparchive/cachedb.{cc,h},writer.{cc,h}: optimizations
(thanks to Anthony Towns)
* apt pdiff support from experimental merged
* apt-pkg/deb/dpkgpm.cc: wording fixes (thanks to Matt Zimmerman)
- fix error in dpkg interaction (closes: #364513, thanks to Martin Dickopp)
|
|
* methods/http.cc: revert pipeline depth and move global Proxy to here
* methods/http.h: remove Proxy from here
|
