summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2016-08-19CMake: Translations: Don't rebuild .mo for line number changesJulian Andres Klode
If only the line numbers changed in a file without any of the translatable strings changing, the .po and .mo files were rebuilt, making building simple code changes somewhat annoying. We can work around this by passing --add-location=file to msgcomm when we are creating the temporary .pot file used for building the .mo files.
2016-08-18debian: Install etc/apt if present (e.g., on Ubuntu)Julian Andres Klode
On Ubuntu, cmake installs are vendor-specific apt.conf.d snippet, causing the build to fail.
2016-08-17Release 1.3~rc21.3_rc2Julian Andres Klode
2016-08-17methods: read config in most to least specific orderDavid Kalnischkies
The implementation of the generic config fallback did the fallback in the wrong order so that the least specific option wasn't the last value picked but in fact the first one… doh! So in the bugreports case http -> https -> http::<hostname> -> https::<hostname> while it should have been the reverse as before. Regression-In: 30060442025824c491f58887ca7369f3c572fa57 Closes: 834642
2016-08-17don't count each Type as an individual deb822-sources stanzaDavid Kalnischkies
Reported-By: Mattia Rizzolo <mattia@debian.org> in #834629
2016-08-17add --with-source option and Packages/Sources supportDavid Kalnischkies
We support "./foobar.deb" as a way to install a deb file directly. Recently .changes files were added. This highlights a problem as you can't add the changes file without also trying to install all of them. Now, it could also be handy to add entire Packages/Sources files to perhaps get a bunch of packages in without installing them all implicitly. This commit introduces --with-source which allows to add *.deb, *.changes, *.dsc, source-dirs, Packages & Sources files (the later can also be compressed) without also installing them.
2016-08-17allow spaces in fingerprints for 'apt-key del'David Kalnischkies
Fingerprints tend to be displayed in space-separated octet pairs so be nice and allow delete to remove a key based on such a string rather than requiring that the user is deleting all the spaces manually.
2016-08-17add the gpg-classic variant to the gpgv/gnupg or-groupDavid Kalnischkies
We need to support partial upgrades anyhow, so we have to deal with the different versions and your tests try to ensure that we do, so we shouldn't make any explicit higher requirements.
2016-08-17tests: update self-sigs on our test keysDavid Kalnischkies
gpg upstream committed "gpgv: Tweak default options for extra security." applied on the 1.x and 2.x branches: http://git.gnupg.org/cgi-bin/gitweb.cgi?p=gnupg.git;a=commit;h=e32c575e0f3704e7563048eea6d26844bdfc494b This commit includes "[…], but we should validate the key by its self signature for primary key, and back signature for subkey." Our testkeys are old and do not really considered best practices in the last years, so their most recent self-signature is SHA1-only which with this gpg commit and our testcases defaulting to --weak-digest sha1 are refused. So what we do here is just applying some of the recent best practices on top of our testcase keys. Gbp-Dch: Ignore
2016-08-17tests: set source directory for gdbDavid Kalnischkies
Helps interactive gdb calls find the source code. Gbp-Dch: Ignore
2016-08-17default to Dir=/ in dpkg/status file finding magicDavid Kalnischkies
Seen in cme #833656 if Dir isn't set (yet) we end up later absoluting a path which was supposed to be absolute already, so if Dir is empty we assume it to be '/' instead. In practice this is a bug in the software using libapt, but for maxium compatibility lets explicitly set the default value here to be safe. Reported-By: Paul Wise <pabs@debian.org> Inspired-By: Brendan O'Dea <bod@debian.org> Fixes-Regression: 475f75506db48a7fa90711fce4ed129f6a14cc9a Shadows-Bug: #833656
2016-08-17support compression and by-hash for .diff/Index filesDavid Kalnischkies
In af81ab9030229b4ce6cbe28f0f0831d4896fda01 by-hash got implemented as a special compression type for our usual index files like Packages. Missing in this scheme was the special .diff/Index index file containing the info about individual patches for this index file. Deriving from the index file class directly we inherent the compression handling infrastructure and in this way also by-hash nearly for free. Closes: #824926
2016-08-17support getting only-uncompressed files via by-hashDavid Kalnischkies
The URI we later want to modify to get the file via by-hash was unset in case a file was only available uncompressed (which is usually not the case) causing an acquire error.
2016-08-17set the correct item FileSize in by-hash caseDavid Kalnischkies
In af81ab9030229b4ce6cbe28f0f0831d4896fda01 we implement by-hash as a special compression type, which breaks this filesize setting as the code is looking for a foobar.by-hash file then. Dealing this slightly gets us the intended value. Note that this has no direct effect as this value will be set in other ways, too, and could only effect progress reporting. Gbp-Dch: Ignore
2016-08-17retry without same redirection mirror on 404 errorsDavid Kalnischkies
If 9b8034a9fd40b4d05075fda719e61f6eb4c45678 serves the Release files from a partial mirror we will end up getting 404 for some of the indexes. Instead of giving up, we will instead ignore our same redirection mirror constrain and ask the redirection service as a potential hashsum mismatch is better than keeping the certain 404 error.
2016-08-17check internal redirections for loops, tooDavid Kalnischkies
Now that we have the redirections loopchecker centrally in our items we can use it also to prevent internal redirections to loop caused by bugs as in a few instances we get into the business of rewriting the URI we will query by ourself as we predict we would see such a redirect anyway. Our code has no bugs of course, hence no practical difference. ;) Gbp-Dch: Ignore
2016-08-16log with the failed item description, not with next tryDavid Kalnischkies
The failure handling frequently changes URI & Description of the failed item to try a slightly different combination which might work, but the logging of the failure happens only afterwards as the same failure handling decides if this is a critical error or not so we need a backup here instead of potentially new content. A purely cosmetic issue, but can still be confusing for humans.
2016-08-16don't try pipelining if server closes connectionsDavid Kalnischkies
If a server closes a connection after sending us a file that tends to mean that its a type of server who always closes the connection – it is therefore relatively pointless to try pipelining with it even if it isn't a problem by itself: apt is just restarting the pipeline each time after it got served one file and the connection is closed. The problem starts if one or more proxies are between the server and apt and they disagree about how the connection should be as in the bugreporters case where the responses apt gets contain both Keep-Alive and Proxy-Connection headers (which apt both ignores) indicating a proxy is trying to keep a connection open while the response also contains "Connection: close" indicating the opposite which apt understands and respects as it is required to do. We avoid stepping into this abyss by not performing pipelining anymore if we got a respond with the indication to close connection if the response was otherwise a success – error messages are sent by some servers via this method as their pages tend to be created dynamically and hence their size isn't known a priori to them. Closes: #832113
2016-08-16don't sent Range requests if we know its not acceptedDavid Kalnischkies
If the server told us in a previous request that it isn't supporting Ranges with bytes via an Accept-Ranges header missing bytes, we don't try to formulate requests using Ranges.
2016-08-16reorganize server-states resetting in http/httpsDavid Kalnischkies
We keep various information bits about the server around, some only effecting the currently handled file (like sizes) while others should be persistent (like pipeline detections). http used to reset all file-related manually, which is a bit silly if we already have a Reset() method – which does reset all through –, so extending it with a parameter for reuse and calling it from https too (as this was previously resetting by just creating a new state struct – it uses no value of the persistent state-keeping yet as it supports no pipelining). Gbp-Dch: Ignore
2016-08-15CMake: Install bash completions via cmakeJulian Andres Klode
Having the completions installed only by the packaging was an oversight. Gbp-Dch: ignore
2016-08-13Change anonscm.d.o links to /git/apt/apt.git and httpsJulian Andres Klode
This also fixes Debian/apt#20, but is slightly more complete. I think /git also looks better than /cgit, so let's switch the Vcs entry in control over too.
2016-08-13http(s): allow empty values for header fieldsDavid Kalnischkies
It seems completely pointless from a server-POV to sent empty header fields, so most of them don't do it (simply proven by this limitation existing since day one) – but it is technically allowed by the RFC as the surounding whitespaces are optional and Github seems to like sending "X-Geo-Block-List:\r\n" since recently (bug reports in other http clients indicate July) at least sometimes as the reporter claims to have seen it on https only even through it can happen with both. Closes: 834048
2016-08-12drop incorrect const attribute from DirectoryExistsDavid Kalnischkies
Since its existence in 2010 DirectoryExists was always marked with this attribute, but for no real reason. Arguably a check for the existence of the file is not modifying global state, so theoretically this shouldn't be a problem. It is wrong from a logical point of view through as between two calls the directory could be created so the promise we made to the compiler that it could remove the second call would be wrong, so API wise it is wrong. It's a bit mysterious that this is only observeable on ppc64el and can be fixed by reordering code ever so slightly, but in the end its more our fault for adding this attribute than the compilers fault for doing something silly based on the attribute. LP: 1473674
2016-08-12fileutl: empty file support: Avoid fstat() on -1 fd and check resultJulian Andres Klode
When checking if a file is empty, we forget to check that fstat() actually worked.
2016-08-12tests: don't do boundless string compares with data()David Kalnischkies
Git-Dch: Ignore
2016-08-12ensure a good clock() value for usage and testsDavid Kalnischkies
We use clock() as a very cheap way of getting a "random" value, but the manpage warns that this could return -1, so we should be dealing with this. Additionally, e.g. on hurd-i386 the value increases only slowly – to slow for our fast running tests for randomness hence producing the same range in both samples, so we introduce a simple busy-wait loop (as clock is counting processor time used by the program) in the test which delays the second sample just enough making our randomness a bit more predictable.
2016-08-12don't perform int<float in progress bar drawingDavid Kalnischkies
Comparing floating numbers is always fun and in this instance a 9 < 9.0 is "somehow" true on hurd-i386 letting the tests fail by reporting that too much progress achieved. A bit mysterious, but with some rework we can use code which avoids dealing with the floats in this way entirely and make our testcases happy.
2016-08-12ctest: show test output in case of failuresDavid Kalnischkies
ctest as run by cmake by default does not show the output of the tests even if the tests failed. In terms of our tests it could be handy to set it always, but unfortunately it seems like cmake doesn't allow it if the internet is to be believed, so lets enable it at least while building packages and on travis. Gbp-Dch: Ignore
2016-08-11CMake: Exclude .md5 and .map doxygen files from installJulian Andres Klode
This is much better than removing them in debian/rules. Gbp-Dch: ignore
2016-08-11CMake: Use COPYONLY instead of @ONLYJulian Andres Klode
I don't know what happened back in 2009 when I wrote this, but it seems I used the wrong option. These files should not have any variable substitution done to them.
2016-08-11CMake: Mark Doxygen as required and use DOXYGEN_EXECUTABLEJulian Andres Klode
Seems like I missed that when adding doxygen support. Gbp-Dch: ignore
2016-08-11debian/NEWS: Get rid of 1.3~pre3+cmake1 entryJulian Andres Klode
This was only needed temporarily Thanks: Axel Beckert for reporting
2016-08-11Release 1.3~rc11.3_rc1Julian Andres Klode
This commit looks heavy. Most of that comes from the fact that the ordering of files in the translations changed with the switch to CMake. I could have gone the extra mile to figure out the original ordering and replicate it, but I have chosen to re-order everything by file and line number, as that's easier.
2016-08-11doc/po: Adjust translations for programmatic typo fixJulian Andres Klode
This enhances commit b9e6db821a6ddbc5bf6a90c80c296d4e91283a63. Gbp-Dch: ignore
2016-08-11tests: copy 01autoremove from the right placeDavid Kalnischkies
With cmake using BUILDDIRECTORY at this place is not only as wrong as it was before, but it might not even work always copying the system provided one which might or might not be current and hence fails tests needing it to be current like ./test-apt-move-and-forget-manual-sections We don't want to always use the one from the source directory through either like in autopkgtests. Gbp-Dch: Ignore
2016-08-11Merge branch 'feature/apt-dpkg-comm'David Kalnischkies
2016-08-11Merge branch 'feature/methods'David Kalnischkies
2016-08-11http: auto-configure for local Tor proxy if called as 'tor'David Kalnischkies
With apts http transport supporting socks5h proxies and all the work in terms of configuration of methods based on the name it is called with it becomes surprisingly easy to implement Tor support equally (and perhaps even a bit exceeding) what is available currently in apt-transport-tor. How this will turn out to be handled packaging wise we will see in https://lists.debian.org/deity/2016/08/msg00012.html , but until this is resolved we can add the needed support without actively enabling it for now, so that this can be tested better.
2016-08-11block direct connections to .onion domains (RFC7687)David Kalnischkies
Doing a direct connect to an .onion address (if you don't happen to use it as a local domain, which you shouldn't) is bound to fail and does leak the information that you do use Tor and which hidden service you wanted to connect to to a DNS server. Worse, if the DNS is poisoned and actually resolves tricking a user into believing the setup would work correctly… This does block also the usage of wrappers like torsocks with apt, but with native support available and advertised in the error message this shouldn't really be an issue. Inspired-by: https://bugzilla.mozilla.org/show_bug.cgi?id=1228457
2016-08-10disable explicit configuration of all packages at the endDavid Kalnischkies
With b4450f1dd6bca537e60406b2383ab154a3e1485f we dropped what we calculated here later on and now that we don't need it in the meantime either we can just skip the busy work by default and expect dpkg to do the right thing dropping also our little "last explicit configures" removal trick introduced in b4450f1dd6bca537e60406b2383ab154a3e1485f. This enables the last of a bunch of previously experimental options, some of them existing still, but are very special and hence not really worth documenting anymore (especially as it would need to be rewritten now entirely) which is why the documentation is nearly completely dropped. The order of configuration stanzas in the simulation code changes slightly as it isn't concerning itself with finding the 'right' order, but any order is valid anyhow as long as the entire set happens in the same call.
2016-08-10simulate all package manager actions explicitlyDavid Kalnischkies
If a planner lets actions to be figured out by dpkg in pending calls these actions aren't mentioned in a simulation. While that might be a good thing for debugging, it would be a change in behavior and especially if a planner avoids explicit removals could be confusing for users. As such we perform the same 'trick' as in the dpkg implementation by performing explicitly what would be done by the pending calls. To save us some work and avoid desyncs we perform a layer violation by using deb/ code in the generic simulation – and further we perform ugly dynamic_cast to avoid breaking the ABI for nothing; aptitude is the only other user of the simulation class according to codesearch.d.n and for that our little trick works. It just isn't working if you happen to extend pkgSimulate or otherwise manage to call the protected Go methods directly – which isn't very realistic/practical.
2016-08-10try to avoid removal of crossgraded packagesDavid Kalnischkies
The user has to approve the removal of a crossgraded package as it might be needed to remove it (temporarily) in the process, but in most cases we can happily avoid it and let dpkg unpack over it skipping the remove. This has some effects on progress reporting and how deal with selections through which makes this a tiny bit complicated.
2016-08-10allow user@host (aka: no password) in URI parsingDavid Kalnischkies
If the URI had no password the username was ignored
2016-08-10fix programmatic typo in apt-key manpageDavid Kalnischkies
Reported-By: Johannes 'josch' Schauer on IRC Gbp-Dch: Ignore
2016-08-10allow methods to be disabled and redirected via configDavid Kalnischkies
To prevent accidents like adding http-sources while using tor+http it can make sense to allow disabling methods. It might even make sense to allow "redirections" and adding "symlinked" methods via configuration. This could e.g. allow using different options for certain sources by adding and configuring a "virtual" new method which picks up the config based on the name it was called with like e.g. http does if called as tor+http.
2016-08-10implement socks5h proxy support for http methodDavid Kalnischkies
Socks support is a requested feature in sofar that the internet is actually believing Acquire::socks::Proxy would exist. It doesn't and this commit isn't adding it as that isn't how our configuration works, but it allows Acquire::http::Proxy="socks5h://…". The HTTPS method was changed already to support socks proxies (all versions) via curl. This commit implements only SOCKS5 (RFC1928) with no auth or pass&user auth (RFC1929), but not GSSAPI which is required by the RFC. The 'h' in the protocol name further indicates that DNS resolution is delegated to the socks proxy rather than performed locally. The implementation works and was tested with Tor as socks proxy for which implementing socks5h only can actually be considered a feature. Closes: 744934
2016-08-10implement generic config fallback for methodsDavid Kalnischkies
The https method implemented for a long while now a hardcoded fallback to the same options in http, which, while it works, is rather inflexible if we want to allow the methods to use another name to change their behavior slightly, like apt-transport-tor does to https – most of the diff being s#https#tor#g which then fails to do the full circle fallthrough tor -> https -> http for https sources. With this config infrastructure this could be implemented now.
2016-08-10use the same redirection handling for http and httpsDavid Kalnischkies
cURL which backs our https implementation can handle redirects on its own, but by dealing with them on our own we gain finer control over which redirections will be performed (we don't like https → http) and by whom so that redirections to other hosts correctly spawn a new https method dealing with these instead of letting the current one deal with it.
2016-08-10detect redirection loops in acquire instead of workersDavid Kalnischkies
Having the detection handled in specific (http) workers means that a redirection loop over different hostnames isn't detected. Its also not a good idea have this implement in each method independently even if it would work