summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2017-02-22get pdiff files from the same mirror as the indexDavid Kalnischkies
In ad9416611ab83f7799f2dcb4bf7f3ef30e9fe6f8 we fall back to asking the original mirror (e.g. a redirector) if we do not get the expected result. This works for the indexes, but patches are a different beast and much simpler. Adding this fallback code here seems like overkill as they are usually right along their Index file, so actually forward the relevant settings to the patch items which fixes pdiff support combined with a redirector and partial mirrors as in such a situation the pdiff patches would be 404 and the complete index would be downloaded. (cherry picked from commit 5832913a49d4f7c75527264a935cc0ce00627f1d)
2017-02-22skip unconfigure for unconfigured to-be removed pkgsDavid Kalnischkies
(cherry picked from commit 8e7a99564dd57b0dcb7df47b43e71ccefc8e0ebe)
2017-02-22do not configure unconfigured to be removed packagesDavid Kalnischkies
We try to configure all packages at the end which need to be configured, but that also applies to packages which weren't completely installed (e.g. maintainerscript failed) we end up removing in this interaction instead. APT doesn't perform this explicit configure in the end as it is using "dpkg --configure --pending", but it does confuse the progress report and potentially also hook scripts. Regression-Of: 9ffbac99e52c91182ed8ff8678a994626b194e69 (cherry picked from commit bb9c5972524ac5c078fa0f0bc5674c7a0fe01fb4)
2017-02-22don't perform implicit crossgrades involving M-A:sameDavid Kalnischkies
dpkg stumbles over these (#844300) and we haven't dropped 'easier' removes to be implicit and to be scheduled by dpkg by default so far so we shouldn't push the decision in such cases to dpkg either. (cherry picked from commit 53f3fc59f4eb37eea57bbde53fb75f2e15af0378)
2017-02-22improve arch-unqualified dpkg-progress parsingDavid Kalnischkies
Our old idea was to look for the first package which would be "touched" and take this as the package dpkg is talking about, but that is incorrect in complicated situations like a package upgraded to/from multiple M-A:same siblings installed. As we us the progress report to decide what is still needed we have to be reasonabily right about the package dpkg is talking about, so we jump to quite a few loops to get it. (cherry picked from commit 4b10240cca0dc0a4e82e42959545d2ae7e622d29)
2017-02-22correct cross & disappear progress detectionDavid Kalnischkies
Given that we use the progress information to skip over actions dpkg has already done like not purging a package which was already removed and had no config files or not acting on disappeared packages and such it is important that apt and dpkg agree on which states the package has to pass through. To ensure that we keep tabs on this in the future a warning is added at the end if apt hasn't seen all the action it was supposed to see. I can't wait for the first bugreporters to wonder about this… (cherry picked from commit dabe9e2482180ada77d2adda2b3c03db22059fb8)
2017-02-22react to trig-pend only if we have nothing else to doDavid Kalnischkies
If a package is triggered dpkg frequently issues two messages about it causing us to make a note about it both times which messes up our planned dpkg actions view. Adding these actions if we have nothing else planned fixes this and should still be correct as those planned actions will deal with the triggering just fine and we avoid strange problems like a package triggered before its removed… (cherry picked from commit 066d4a5bab628ef8220971bb5763ff8f3a13de07)
2017-02-22http: clear content before reporting the failureEdgar Fuß
[Comment from commiter:] I have the feeling that the issue itself is fixed for a while already as nowadays we have testcases involving a webserver closing the connection on error (look for "closeOnError") and no even remotely recent reports about it, but moving the content clearance above the failure report is a valid change and shouldn't hurt. Closes: #465572 (cherry picked from commit 324bb34d77a43d1be411c402b2e11f588194439a)
2017-02-22add TMP/TEMP/TEMPDIR to the TMPDIR DropPrivileges danceDavid Kalnischkies
apt tools do not really support these other variables, but tools apt calls might, so lets play save and clean those up as needed. Reported-By: Paul Wise (pabs) on IRC (cherry picked from commit e2c8c825a5470e33c25d00e07de188d0e03922c8)
2017-02-22reset HOME, USER(NAME), TMPDIR & SHELL in DropPrivilegesDavid Kalnischkies
We can't cleanup the environment like e.g. sudo would do as you usually want the environment to "leak" into these helpers, but some variables like HOME should really not have still the value of the root user – it could confuse the helpers (USER) and HOME isn't accessible anyhow. Closes: 842877 (cherry picked from commit 34b491e735ad47c4805e63f3b83a659b8d10262b)
2017-02-22keep Release.gpg on untrusted to trusted IMS-HitDavid Kalnischkies
A user relying on the deprecated behaviour of apt-get to accept a source with an unknown pubkey to install a package containing the key expects that the following 'apt-get update' causes the source to be considered as trusted, but in case the source hadn't changed in the meantime this wasn't happening: The source kept being untrusted until the Release file was changed. This only effects sources not using InRelease and only apt-get, the apt binary downright refuses this course of actions, but it is a common way of adding external sources. Closes: 838779 (cherry picked from commit 84eec207be35b8c117c430296d4c212b079c00c1) LP: #1657440
2017-02-22don't install new deps of candidates for kept back pkgsDavid Kalnischkies
In effect this is an extension of the 6 years old commit a8dfff90aa740889eb99d00fde5d70908d9fd88a which uses the autoremover to remove packages again from the solution which are no longer needed to be there. Commonly these are dependencies of packages we end up not installed due to problem resolver decisions. Slightly less common is the situation we deal with here: a package which we wanted to upgrade sporting a new dependency, but ended up holding back. The problem is that all versions of an installed reverse dependencies can bring back a "garbage" package – we need to do this as there is nothing inherently wrong in having garbage packages installed or upgrade them, which itself would have garbage dependencies, so just blindly killing all new garbage packages would prevent the upgrade (and actually generate errors). What we should be doing is looking only at the version we will have on the system, disregarding all old/new reverse dependencies. Reported-By: Stuart Prescott (themill) on IRC (cherry picked from commit 952171787a0b865c17d5c9476e272106383ae93a)
2017-01-17Release 1.3.41.3.4Julian Andres Klode
2017-01-17test: use downloadfile instead of apthelper download-fileJulian Andres Klode
This prevents CI failures from happening in 1.3 and 1.2 and might actually be more complete. Gbp-Dch: ignore (cherry picked from commit 803dabde5a4345ce83b3d2ffbd475786db9769d9)
2017-01-17https: Quote path in URL before passing it to curlJulian Andres Klode
Curl requires URLs to be urlencoded. We are however giving it undecoded URLs. This causes it go completely nuts if there is a space in the URI, producing requests like: GET /a file HTTP/1.1 which the servers then interpret as a GET request for "/a" with HTTP version "file" or some other non-sense. This works around the issue by encoding the path component of the URL. I'm not sure if we should encode other parts of the URL as well, this one seems to do the trick for the actual issue at hand. A more correct fix is to avoid the dequoting and (re-)quoting of URLs when a redirect occurs / a new request is sent. That's been on the radar for probably a year or two now, but nobody bothered implementing that yet. LP: #1651923 (cherry picked from commit 994515e689dcc5f963f5fed58284831750a5da03)
2016-12-14Release 1.3.31.3.3Julian Andres Klode
2016-12-14Do not (re)start "apt-daily.system"Michael Vogt
This unit runs unattended-upgrades. If apt itself is part of the upgrade a restart of the unit will kill unattended-upgrades. This will lead to an inconsistent dpkg status. Closes: #841763 Thanks: Alexandre Detiste (cherry picked from commit e133bb5e81b10bf059b3abeab2d9e41f7206e446) LP: #1649959
2016-12-08Release 1.3.2 security update1.3.2Julian Andres Klode
2016-12-08gpgv: Flush the files before checking for errorsJulian Andres Klode
This is a follow up to the previous issue where we did not check if getline() returned -1 due to an end of file or due to an error like memory allocation, treating both as end of file. Here we ensure that we also handle buffered writes correctly by flushing the files before checking for any errors in our error stack. Buffered writes themselves were introduced in 1.1.9, but the function was never called with a buffered file from inside apt until commit 46c4043d741cb2c1d54e7f5bfaa234f1b7580f6c which was first released with apt 1.2.10. The function is public, though, so fixing this is a good idea anyway. Affected: >= 1.1.9 (cherry picked from commit 6212ee84a517ed68217429022bd45c108ecf9f85)
2016-12-08SECURITY UPDATE: gpgv: Check for errors when splitting files (CVE-2016-1252)Julian Andres Klode
This fixes a security issue where signatures of the InRelease files could be circumvented in a man-in-the-middle attack, giving attackers the ability to serve any packages they want to a system, in turn giving them root access. It turns out that getline() may not only return EINVAL as stated in the documentation - it might also return in case of an error when allocating memory. This fix not only adds a check that reading worked correctly, it also implicitly checks that all writes worked by reporting any other error that occurred inside the loop and was logged by apt. Affected: >= 0.9.8 Reported-By: Jann Horn <jannh@google.com> Thanks: Jann Horn, Google Project Zero for reporting the issue LP: #1647467 (cherry picked from commit 51be550c5c38a2e1ddfc2af50a9fab73ccf78026)
2016-12-08Hack around Ubuntu rejecting symlink copyrightJulian Andres Klode
2016-12-08debian/gbp.conf: Update for 1.3.y branchJulian Andres Klode
Gbp-Dch: ignore
2016-10-04Release 1.3.11.3.1Julian Andres Klode
2016-10-04travis: Move codecov from after_success to after_scriptJulian Andres Klode
We always want to run codecov test, even if there are spurious failures. We should really work around those failures more, though, it is starting to annoy me.
2016-10-04Turkish program translation updateMert Dirik
Closes: #838731
2016-10-04Japanese manpages translation updatevictory
2016-10-04Do not read stderr from proxy autodetection scriptsJulian Andres Klode
This fixes a regression introduced in commit 8f858d560e3b7b475c623c4e242d1edce246025a don't leak FD in AutoProxyDetect command return parsing which accidentally made the proxy autodetection code also read the scripts output on stderr, not only on stdout when it switched the code from popen() to Popen(). Reported-By: Tim Small <tim@seoss.co.uk>
2016-09-20Release 1.31.3Julian Andres Klode
2016-09-18VersionHash: Do not skip too long dependency linesJulian Andres Klode
If the dependency line does not contain spaces in the repository but does in the dpkg status file (because dpkg normalized the dependency list), the dpkg line might be longer than the line in the repository. If it now happens to be longer than 1024 characters, it would be skipped, causing the hashes to be out of date. Note that we have to bump the minor cache version again as this changes the format slightly, and we might get mismatches with an older src cache otherwise. Fixes Debian/apt#23
2016-09-11travis: Add coverage testing using codecov.ioJulian Andres Klode
This allows fully automated code coverage testing, which is basically awesome. To allow the methods and solvers and stuff which run as _apt to write to our build directory, we need to adjust the permissions a bit, but otherwise it's OK. Gbp-Dch: ignore
2016-09-11Coverage: Do not print messages from gcovJulian Andres Klode
We need to ignore messages from gcov. All those messages start with profiling: and are printed using vfprintf(), so the only thing we can do is add a library overriding those functions and linking apt-pkg to it.
2016-09-11CMake: Add coverage build typeJulian Andres Klode
This allows us to easily test coverage
2016-09-11travis: use ninja and ccache for buildingJulian Andres Klode
This cleans up the output a bit, it should also improve performance, but unfortunately, this does not really seem to be the case. Gbp-Dch: ignore
2016-09-07test: Always install dpkg into our tests, regardless of MAJulian Andres Klode
Even if we only configure a single architecture, install dpkg, so dpkg can assert multi arch correctly. This also has the nice side effect of making single architecture and multiple architecture test cases more uniform. This fixes a regression from f878d3a862128bc1385616751ae1d78246b1bd01 ("test: Assert multi-arch in the chroot").
2016-09-07test: framework: Ensure copied status files have trailing linesJulian Andres Klode
If we copied one of the existing status files, we might not have a trailing newline, so let's add one. Gbp-Dch: ignore
2016-09-07edsp: try 2 to read responses even if writing failedDavid Kalnischkies
Commit b60c8a89c281f2bb945d426d2215cbf8f5760738 improved the situation, but due to inconsistency mostly for planners, not for solvers. As the idea of hiding errors if we show another error is a bit scary (as the extern error might be a followup of our intern error, rather than the reason for our intern error as it is at the moment) we don't discard the errors, but if we got an extern error we show them directly removing them from the error list at the end of the run – that list will contain the extern error which hopefully gives us the best of both worlds. The problem itself is the same as before: The externals exiting before apt is done talking to them. Reported-By: Johannes 'josch' Schauer on IRC
2016-09-04abort connection on '.' target replies in SRVDavid Kalnischkies
Commit 3af3ac2f5ec007badeded46a94be2bd06b9917a2 (released in 1.3~pre1) implements proper fallback for SRV, but that works actually too good as the RFC defines that such an SRV record should indicate that the server doesn't provide this service and apt should respect this. The solution is hence to fail again as requested even if that isn't what the user (and perhaps even the server admins) wanted. At least we will print a message now explicitly mentioning SRV to point people in the right direction. Reported-In: https://bugs.kali.org/view.php?id=3525 Reported-By: Raphaël Hertzog
2016-09-03update testing hints in README.mdDavid Kalnischkies
Gbp-Dch: Ignore
2016-09-02Release 1.3~rc41.3_rc4Julian Andres Klode
2016-09-02debian: Pass -O to make to get readable build logsJulian Andres Klode
Normally make just lets every job write its output directly, making the log fairly hard to read with high concurrency.
2016-09-02debian: Move bugscript to old location for overlayfs xdev issueJulian Andres Klode
dpkg on overlayfs cannot rename apt/script to apt, because overlayfs will not let it move apt to a backup name, responding with XDEV instead.
2016-09-02acquire: Use priority queues and a 3 stage pipeline designJulian Andres Klode
Employ a priority queue instead of a normal queue to hold the items; and only add items to the running pipeline if their priority is the same or higher than the priority of items in the queue. The priorities are designed for a 3 stage pipeline system: In stage 1, all Release files and .diff/Index files are fetched. This allows us to determine what files remain to be fetched, and thus ensures a usable progress reporting. In stage 2, all Pdiff patches are fetched, so we can apply them in parallel with fetching other files in stage 3. In stage 3, all other files are fetched (complete index files such as Contents, Packages). Performance improvements, mainly from fetching the pdiff patches before complete files, so they can be applied in parallel: For the 01 Sep 2016 03:35:23 UTC -> 02 Sep 2016 09:25:37 update of Debian unstable and testing with Contents and appstream for amd64 and i386, update time reduced from 37 seconds to 24-28 seconds. Previously, apt would first download new DEP11 icon tarballs and metadata files, causing the CPU to be idle. By fetching the diffs in stage 2, we can now patch our contents and Packages files while we are downloading the DEP11 stuff.
2016-09-02CMake: apt-pkg: Use correct ICONV_INCLUDE_DIRS variableJulian Andres Klode
This accidentally used ICONV_DIRECTORIES, which does not even exist. Weird.
2016-09-02CMake: test/libapt: Use a prebuilt GTest library if availableJulian Andres Klode
If a non-existing source directory is specified, try finding the system gtest library. Debian derived distributions are a bit strange because they only ship the source code and not the library...
2016-09-02gitignore: Add generated docbook stylesheetsJulian Andres Klode
I switched them over to generated files in commit 9fb81c6e54a2fe05c0ad0b877fd32f30358e3877, but forgot to add them to the ignore file. Gbp-Dch: ignore
2016-09-01support long keyid and fingerprint in gpgv's GOODSIGDavid Kalnischkies
In gpgv1 GOODSIG (and the other messages of status-fd) are documented as sending the long keyid. In gpgv2 it is documented to be either long keyid or the fingerprint. At the moment it is still the long keyid, but the documentation hints at the possibility of changing this. We care about this for Signed-By support as we detect this way if the right fingerprint has signed this file (or not). The check itself is done via VALIDSIG which always is a fingerprint, but there must also be a GOODSIG (as expired sigs are valid, too) found to be accepted which wouldn't be found in the fingerprint-case and the signature hence refused.
2016-09-01re-add apt breaks/replaces apt-utils (<< 1.3~exp2~)David Kalnischkies
The recently added (increased actually) Breaks were accidently dropped while our set of mostly old and outdated breaks was cleaned up. Regression-From: 20d2f4a4f164cd9026dad698e471c95d7c28973b Previously-Add-In: ab07af708e49c9219940ffd3e20a01c763267e03 Closes: #836220
2016-09-01tests: silence -Wmissing-declarationsDavid Kalnischkies
Gbp-Dch: Ignore Reported-By: gcc -Wmissing-declarations
2016-09-01try not to call memcpy with length 0 in hash calculationsDavid Kalnischkies
memcpy is marked as nonnull for its input, but ignores the input anyhow if the declared length is zero. Our SHA2 implementations do this as well, it was "just" MD5 and SHA1 missing, so we add the length check here as well as along the callstack as it is really pointless to do all these method calls for "nothing". Reported-By: gcc -fsanitize=undefined
2016-09-01test-apt-cdrom: Fix for gnupg 2.1.15Julian Andres Klode
gpg annoyingly changed its output and broke our test suite again by adding two extra lines about key type and issuer. Really annoying. Those lines also have more than one space after the colon, so let's use \s* there - and also change the other lines to support variable length whitespace in case gpg decides to break things there too.