| Age | Commit message (Collapse) | Author |
|---|
|
If apt renames a file to .FAILED it leaves its namespace and is never
touched again – expect since 1.1~exp4 in which "apt clean" will remove
those files. The usefulness of these files rapidly degrades if you don't
keep the update log itself (together with debug output in the best case)
through and on 99% of all system they will be kept around forever just
to collect dust over time and eat up space.
With this commit an update call will remove all FAILED files of previous
runs, so that the FAILED files you have on disk are always only the ones
related to the last apt run stopping apt from hoarding files.
Closes: 846476
(cherry picked from commit 7ca83492e802967f183babf06ab541b1b51f1703)
|
|
The mode wasn't working at all if not used together with --fix-missing
which while likely to come in pairs its legal to use standalone.
Regression-in: eb1f04dda07c2b69549ad9fd793cca0e91841b3e
(cherry picked from commit 3313eaf97c83177433478505c05815ab02f9782b)
|
|
Unlikely that anyone is actually running into this, but if we asked to
not generate a cache and avoid it in the first step we shouldn't create
one implicitly anyway by displaying the statistics.
(cherry picked from commit 33f982b90a4f77be18cb82daf8c79e9c5513761c)
|
|
The crude way of preparing a message to be a multiline value failed at
generation valid deb822 in case the error message ended with a new line
like the resolving errors from apt do. apt itself can parse these, but
other tools like grep-dctrl choke on it, so be nice and print valid.
Reported-By: Johannes 'josch' Schauer on IRC
(cherry picked from commit 0161280405fe5aa256dc9df6a56106dd3a1a6f38)
|
|
That was the case already for tar-only and diff-only, but in a more
confusing way and without a message while dsc "worked" before resulting
in a dpkg-source error shortly after as tar/diff files aren't available…
(cherry picked from commit 58ebb3017baf46e33a9bb2c1779d6daede27d108)
|
|
dpkg stumbles over these (#844300) and we haven't dropped 'easier'
removes to be implicit and to be scheduled by dpkg by default so far
so we shouldn't push the decision in such cases to dpkg either.
(cherry picked from commit 53f3fc59f4eb37eea57bbde53fb75f2e15af0378)
|
|
Our old idea was to look for the first package which would be "touched"
and take this as the package dpkg is talking about, but that is
incorrect in complicated situations like a package upgraded to/from
multiple M-A:same siblings installed.
As we us the progress report to decide what is still needed we have to
be reasonabily right about the package dpkg is talking about, so we jump
to quite a few loops to get it.
(cherry picked from commit 4b10240cca0dc0a4e82e42959545d2ae7e622d29)
|
|
Given that we use the progress information to skip over actions dpkg has
already done like not purging a package which was already removed and
had no config files or not acting on disappeared packages and such it is
important that apt and dpkg agree on which states the package has to
pass through.
To ensure that we keep tabs on this in the future a warning is added at
the end if apt hasn't seen all the action it was supposed to see. I
can't wait for the first bugreporters to wonder about this…
(cherry picked from commit dabe9e2482180ada77d2adda2b3c03db22059fb8)
|
|
We can't cleanup the environment like e.g. sudo would do as you usually
want the environment to "leak" into these helpers, but some variables
like HOME should really not have still the value of the root user – it
could confuse the helpers (USER) and HOME isn't accessible anyhow.
Closes: 842877
(cherry picked from commit 34b491e735ad47c4805e63f3b83a659b8d10262b)
|
|
A user relying on the deprecated behaviour of apt-get to accept a source
with an unknown pubkey to install a package containing the key expects
that the following 'apt-get update' causes the source to be considered
as trusted, but in case the source hadn't changed in the meantime this
wasn't happening: The source kept being untrusted until the Release file
was changed.
This only effects sources not using InRelease and only apt-get, the apt
binary downright refuses this course of actions, but it is a common way
of adding external sources.
Closes: 838779
(cherry picked from commit 84eec207be35b8c117c430296d4c212b079c00c1)
LP: #1657440
|
|
In effect this is an extension of the 6 years old commit
a8dfff90aa740889eb99d00fde5d70908d9fd88a which uses the autoremover to
remove packages again from the solution which are no longer needed to be
there. Commonly these are dependencies of packages we end up not
installed due to problem resolver decisions. Slightly less common is
the situation we deal with here: a package which we wanted to upgrade
sporting a new dependency, but ended up holding back.
The problem is that all versions of an installed reverse dependencies can
bring back a "garbage" package – we need to do this as there is
nothing inherently wrong in having garbage packages installed or upgrade
them, which itself would have garbage dependencies, so just blindly
killing all new garbage packages would prevent the upgrade (and actually
generate errors). What we should be doing is looking only at the version
we will have on the system, disregarding all old/new reverse dependencies.
Reported-By: Stuart Prescott (themill) on IRC
(cherry picked from commit 952171787a0b865c17d5c9476e272106383ae93a)
|
|
This prevents CI failures from happening in 1.3 and 1.2 and
might actually be more complete.
Gbp-Dch: ignore
(cherry picked from commit 803dabde5a4345ce83b3d2ffbd475786db9769d9)
|
|
Curl requires URLs to be urlencoded. We are however giving it
undecoded URLs. This causes it go completely nuts if there is
a space in the URI, producing requests like:
GET /a file HTTP/1.1
which the servers then interpret as a GET request for "/a" with
HTTP version "file" or some other non-sense.
This works around the issue by encoding the path component of
the URL. I'm not sure if we should encode other parts of the URL
as well, this one seems to do the trick for the actual issue at
hand.
A more correct fix is to avoid the dequoting and (re-)quoting
of URLs when a redirect occurs / a new request is sent. That's
been on the radar for probably a year or two now, but nobody
bothered implementing that yet.
LP: #1651923
(cherry picked from commit 994515e689dcc5f963f5fed58284831750a5da03)
|
|
If the dependency line does not contain spaces in the repository
but does in the dpkg status file (because dpkg normalized the
dependency list), the dpkg line might be longer than the line
in the repository. If it now happens to be longer than 1024
characters, it would be skipped, causing the hashes to be
out of date.
Note that we have to bump the minor cache version again as
this changes the format slightly, and we might get mismatches
with an older src cache otherwise.
Fixes Debian/apt#23
|
|
Even if we only configure a single architecture, install dpkg, so
dpkg can assert multi arch correctly. This also has the nice side
effect of making single architecture and multiple architecture
test cases more uniform.
This fixes a regression from f878d3a862128bc1385616751ae1d78246b1bd01
("test: Assert multi-arch in the chroot").
|
|
If we copied one of the existing status files, we might not have
a trailing newline, so let's add one.
Gbp-Dch: ignore
|
|
Commit b60c8a89c281f2bb945d426d2215cbf8f5760738 improved the situation,
but due to inconsistency mostly for planners, not for solvers. As the
idea of hiding errors if we show another error is a bit scary (as the
extern error might be a followup of our intern error, rather than the
reason for our intern error as it is at the moment) we don't discard the
errors, but if we got an extern error we show them directly removing
them from the error list at the end of the run – that list will contain
the extern error which hopefully gives us the best of both worlds.
The problem itself is the same as before: The externals exiting before
apt is done talking to them.
Reported-By: Johannes 'josch' Schauer on IRC
|
|
Employ a priority queue instead of a normal queue to hold
the items; and only add items to the running pipeline if
their priority is the same or higher than the priority
of items in the queue.
The priorities are designed for a 3 stage pipeline system:
In stage 1, all Release files and .diff/Index files are fetched. This
allows us to determine what files remain to be fetched, and thus
ensures a usable progress reporting.
In stage 2, all Pdiff patches are fetched, so we can apply them
in parallel with fetching other files in stage 3.
In stage 3, all other files are fetched (complete index files
such as Contents, Packages).
Performance improvements, mainly from fetching the pdiff patches
before complete files, so they can be applied in parallel:
For the 01 Sep 2016 03:35:23 UTC -> 02 Sep 2016 09:25:37 update
of Debian unstable and testing with Contents and appstream for
amd64 and i386, update time reduced from 37 seconds to 24-28
seconds.
Previously, apt would first download new DEP11 icon tarballs
and metadata files, causing the CPU to be idle. By fetching
the diffs in stage 2, we can now patch our contents and Packages
files while we are downloading the DEP11 stuff.
|
|
In gpgv1 GOODSIG (and the other messages of status-fd) are documented as
sending the long keyid. In gpgv2 it is documented to be either long
keyid or the fingerprint. At the moment it is still the long keyid, but
the documentation hints at the possibility of changing this.
We care about this for Signed-By support as we detect this way if the
right fingerprint has signed this file (or not). The check itself is
done via VALIDSIG which always is a fingerprint, but there must also be
a GOODSIG (as expired sigs are valid, too) found to be accepted which
wouldn't be found in the fingerprint-case and the signature hence
refused.
|
|
gpg annoyingly changed its output and broke our test suite
again by adding two extra lines about key type and issuer.
Really annoying.
Those lines also have more than one space after the colon,
so let's use \s* there - and also change the other lines to
support variable length whitespace in case gpg decides to
break things there too.
|
|
I actually tried to amend the previous commit, but apparently
I forgot to add the file mode change.
Gbp-Dch: ignore
|
|
If a Binary field contains one or more spaces before a comma, the
code produced a segmentation fault, as it accidentally set a pointer
to 0 instead of the value of the pointer.
If the comma is at the beginning of the field, the code would
create a binStartNext that points one element before the start
of the string, which is undefined behavior.
We also need to check that we do not exit the string during the
replacement of spaces before commas: A string of the form " ,"
would normally exit the boundary of the Buffer:
binStartNext = offset 1 ','
binEnd = offset 0 ' '
isspace_ascii(*binEnd) = true => --binEnd
=> binEnd = - 1
We get rid of the problem by only allowing spaces to be eliminated
if they are not the first character of the buffer:
binStartNext = offset 1 ','
binEnd = offset 0 ' '
binEnd > buffer = false, isspace_ascii(*binEnd) = true
=> exit loop
=> binEnd remains 0
|
|
This works around an issue on Fedora where dpkg complains about
missing build-essential:
dpkg-checkbuilddeps: Unmet build dependencies: build-essential:native
Gbp-Dch: ignore
|
|
Introduce a new -qq mode for our integration test framework,
and make travis use it.
The new -qq mode sets MSGLEVEL to 1. In MSGLEVEL=1, no messages
are generated for passed tests, and all testcase filenames are
printed in the same line.
Also install first in travis, do not ls the installed output
and run the install with chronic, so we only get output if it
failed.
Gbp-Dch: ignore
|
|
An absolute filename for a *.deb file starts with a /. A package with
the name of the file is inserted in the cache which is provided by the
"real" package for internal reasons. The pinning code detects a regex
based wildcard by having the regex start with /. That is no problem
as a / can not be included in a package name… expect that our virtual
filename package can and does.
We fix this two ways actually: First, a regex is only being considered a
regex if it also ends with / (we don't support flags). That stops our
problem with the virtual filename packages already, but to be sure we
also do not enter the loop if matcher and package name are equal.
It has to be noted that the creation of pins for virtual packages like
the here effected filename packages is pointless as only versions can be
pinned, but checking that a package is really purely virtual is too
costly compared to just creating an unused pin.
Closes: 835818
|
|
Without randomizing the order in which we download the index files we
leak needlessly information to the mirrors of which architecture is
native or foreign on this system. More importantly, we leak the order in
which description translations will be used which in most cases will e.g.
have the native tongue first.
Note that the leak effect in practice is limited as apt detects if a file
it wants to download is already available in the latest version from a
previous download and does not query the server in such cases. Combined
with the fact that Translation files are usually updated infrequently
and not all at the same time, so a mirror can never be sure if it got asked
about all files the user wants.
|
|
|
|
On BSD systems, the root group is wheel, not root, so let's
just use the default group here.
Gbp-Dch: ignore
|
|
The host system might not have a dpkg installed, which makes
dpkg fail with:
dpkg not recorded as installed, cannot check for multi-arch support!
That's entirely useless of course. We want to know if dpkg could
support multi-arch in our chroot, so we pseudo-install dpkg into
the chroot and pretend it's version is one version higher than
the minimum dpkg version, so dpkg --assert-multi-arch works on
recent dpkgs.
Gbp-Dch: ignore
|
|
This fixes issues with chroots, but the goal here was to get
the test suite working on systems without dpkg.
|
|
This is needed on FreeBSD which has versions like 11.0-RC1,
otherwise the tests would fail.
|
|
This check should work regardless if dpkg was installed by dpkg
or by a native package manager like RPM or pkg.
Gbp-Dch: ignore
|
|
This is more safe against sticky bits. For example, in FreeBSD
all files created in /tmp have the group set to wheel.
Gbp-Dch: ignore
|
|
Gbp-Dch: ignore
|
|
On BSD systems, we cannot simply use find -name or stuff, we
always have to pass a directory name first.
Gbp-Dch: ignore
|
|
This breaks the tests with FreeBSD's shell, and is not needed -
it works fine without it.
Gbp-Dch: ignore
|
|
Use of echo with special characters is not portable. On a normal
POSIX system, the behavior with backslash escaped strings is
implementation-defined. On an XSI-conformant system, they must
be interpreted.
A way out is the printf command - printf "%b" specifies that
the following argument is to be printed with backslash escapes
interpreted.
Gbp-Dch: ignore
|
|
Especially on non-Debian platforms, dpkg might not list itself
on the host system, and thus dpkg --assert-multi-arch fails.
Gbp-Dch: ignore
|
|
Use /dev/fd in test-bug-712116-dpkg-pre-install-pkgs-hook-multiarch,
skip test-no-fds-leaked-to-maintainer-scripts (it is not guaranteed
that /dev/fd contains all file descriptors), and avoid the unneeded
use of /proc/fd in another test case.
Gbp-Dch: ignore
|
|
Our test suite assumes that dpkg's admindir is var/lib/dpkg. This
might not always be true; for example, on FreeBSD, it is located
at /var/db/dpkg.
Gbp-Dch: ignore
|
|
Gbp-Dch: ignore
|
|
Gbp-Dch: ignore
|
|
That's what it's called on FreeBSD.
Gbp-Dch: ignore
|
|
This is needed for Fedora and FreeBSD.
Gbp-Dch: ignore
|
|
We are simply checking for gnuCMD and gCMD for each command we
are interested in.
Gbp-Dch: ignore
|
|
Gbp-Dch: ignore
|
|
This allows other vendors to use different paths, or to build
your own APT in /opt for testing. Note that this uses + 1 in
some places, as the paths we receive are absolute, but we need
to strip of the initial /.
|
|
Ubuntu uses *.ddeb files for their debug packages, but the interface we
are using since f495992428a396e0f98886c9a761a804aa161c68 to talk to dpkg
isn't supporting *.ddeb files. This used to work previously as apt itself
isn't caring about the filenames at all and if they are explicitly
mentioned dpkg will accept all, too.
It might or might not be a good idea to patch dpkg, too, but regardless
of it happening, we don't want to couple us to closely to dpkg for this
minor feature but testing for this at runtime as it would delay shipping
the fix for the too long commandlines further.
It is also questionable if it is really a good idea to allow any file
extension to be used here (like .foobar in the testcase), but we used to
and we tend to avoid breaking existing usecases if we can help it.
As a bonus, this also allows the installation of ddeb files directly
from the commandline as you can with deb files already. We continue to
ignore udeb through as the user-mistake to useful ratio is too high.
LP: #1616909
|
|
In 105503b4b470c124bc0c271bd8a50e25ecbe9133 we got a warning implemented
for unreadable files which greatly improves the behavior of apt update
already as everything will work as long as we don't need the keys
included in these files. The behavior if they are needed is still
strange through as update will fail claiming missing keys and a manual
test (which the user will likely perform as root) will be successful.
Passing the new warning generated by apt-key through to apt is a bit
strange from an interface point of view, but basically duplicating the
warning code in multiple places doesn't feel right either. That means we
have no translation for the message through as apt-key has no i18n yet.
It also means that if the user has a bunch of sources each of them will
generate a warning for each unreadable file which could result in quite
a few duplicated warnings, but "too many" is better than none.
Closes: 834973
|
|
apt-key has inconsistent behaviour if it can't read a keyring file:
Commands like 'list' skipped silently over such keyrings while 'verify'
failed hard resulting in apt to report cconfusing gpg errors (#834973).
As a first step we teach apt-key to be more consistent here skipping in
all commands over unreadable keyrings, but issuing a warning in the
process, which is as usual for apt commands displayed at the end of the
run.
|
