| Age | Commit message (Collapse) | Author |
|---|
|
Moving the code responsible for parsing the Index file from ::Done into
the slightly earlier ::VerifyDone allows us to still "fail" the download
if we can't make use of the Index for whatever reason, so that the
progress log correctly displays "Ign" instead of "Get" for the file.
This also makes quiet a few debug messages proper error messages (but
those are still hidden by default for Ign lines).
|
|
If we couldn't find an entry for a Sources file we would generate an
error while for a Packages file we would silently skip it due to
assuming it is missing because it is empty. We can do better by checking
if the repository declares that it supports a component we want to get
the file from and if not say so and hint at the user making a typo.
An example were this helps is mozilla.debian.net which dropped the
firefox-aurora component (as upstream did) meaning no upgrades until the
user notices manually that the repository doesn't provide packages
anymore. With this commit warnings are raised hopefully causing the user
to investigate what is wrong (sooner).
|
|
Adopting this change in other frontends will require source changes as
well similar to our own changes in apt-private/.
|
|
|
|
POSIX.1-2008 gives us a range of *at calls to deal with files
including the unlinkat so we can remove a file from a directory
based on a path to the file relative to the directory.
(In our case here the path we have is just the filename)
We avoid changing directories in this way which e.g. fails if the
directory we started in no longer exists or is otherwise inaccessible.
Closes: 860738
|
|
As the proxy commands are not executed as root, a user can run into
permission errors (s)he isn't expecting – as our switching is an
implementation detail – so the error message in that case should really
be better than a generic "error code 100" sending the user in the wrong
direction as that implies the command was executed, but errored out.
Closes: 857885
|
|
Changes nothing on the program front and as the datatypes are
sufficently comparable fixes no bug either, but problems later on if we
ever change the types of those and prevent us using types which are too
large for the values we want to store waste (a tiny bit of) resources.
Gbp-Dch: Ignore
|
|
In complex situations in which we want to unpack a package which has a
conflict/breaks on another package which must be removed due this
conflict apt can decide to perform this remove earlier than initially
planned.
Problem: For three years apt wouldn't remove that package, but the
package which has the conflict… The situation isn't very common and
easily hidden as the package which is removed is unpacked a few actions
later – it becomes visible for packages which protect themselves from
removal through like systemd as the running init resulting in upgrade
failures (#854041).
Note that the package isn't purged, so data shouldn't be lost even if a
user runs into a "hidden" case of it as long as the package sticks to
the policy of removing data only on purge.
Reaching this situation artificially is hard, which is why no testcase
is included, as the situation is highly state dependent. Testing with
"real" systems indicate that slight modifications in the installed
packages set can make the bug not trigger.
Regression-Of: 0eb4af9d3d0c524c7afdc684238aa263ac287449
Thanks: Michael Biebl for helping find this with countless tests
|
|
We need to be able to update 1.4.y in different ways than later
apt versions, and thus need to bump the major version so there
is no collision in the minor version at some point.
|
|
If the last alternative(s) of an Or group is ignored, because it does
not match an architecture list, we would end up keeping the or flag,
effectively making the next AND an OR.
For example, when parsing (on amd64):
debhelper (>= 9), libnacl-dev [amd64] | libnacl-dev [i386]
=> debhelper (>= 9), libnacl-dev |
Which can cause python-apt to crash.
Even worse:
debhelper (>= 9), libnacl-dev [amd64] | libnacl-dev [i386], foobar
=> debhelper (>= 9), libnacl-dev [amd64] | foobar
By setting the previous alternatives Or flag to the current Or flag
if the current alternative is ignored, we solve the issue.
LP: #1694697
|
|
In the intended usecase where this serves as a hack there is no problem
with double/single quotes being present as we write it to a log file
only, but nowadays our calling of apt-key produces a temporary config
file containing this "setting" as well and suddently quoting is
important as the config file syntax is allergic to it.
So the fix is to ignore all quoting whatsoever in the input and just
quote (with singles) the option values with spaces. That gives us 99% of
the time the correct result and the 1% where the quote is an integral
element of the option … doesn't exist – or has bigger problems than a
log file not containing the quote. Same goes for newlines in values.
LP: #1672710
|
|
It says SRCNAME_SRCVER, but the example just gives
the SRCVER part.
Reported-By: Nishanth Aravamudan (nacc) in #ubuntu-devel
|
|
This gets rid of warnings about .ucf-dist files
Reported-By: Axel Beckert (on IRC)
|
|
-1 is not an allowed value for the file descriptor, the only
allowed non-file-descriptor value is AT_FDCWD. So use that
instead.
AT_SYMLINK_NOFOLLOW has a weird semantic: It checks whether
we have the specified access on the symbolic link. It also
is implemented only by glibc on Linux, so it's inherently
non-portable. We should just drop it.
Thanks: James Clarke for debugging these issues
Reported-by: James Clarke <jrtc27@jrtc27.com>
|
|
In the case of build-dep and other commands where a file can be
passed we must make sure not to normalize the path name as that
can have odd side effects, or well, cause the operation to do
nothing.
Test for build-dep-file is adjusted to perform the vcard check
once as "vcard" and once as "VCard", thus testing that this
solves the reported bug.
We inline the std::transform() and optimize it a bit to not
write anything in the common case (package names are defined
to be lowercase, the whole transformation is just for names
that should not exist...) to counter the performance hit of
the added find() call (it's about 0.15% more instructions
than with the existing transform, but we save about 0.67%
in writes...).
Closes: #854794
|
|
Added in dpkg commit 6c8203440bf443d3031ee2ab8485b16c1b6da3b6
|
|
Oh dear, nobody (or rather no tool) saw that yet...
Gbp-Dch: ignore
|
|
Since the introduction of by-hash, two differently named
files might have the same real URL. In our case, the files
icons-64x64.tar.gz and icons-128x128.tar.gz of empty tarballs.
APT would try to merge them and end with weird errors because
it completed the first download and enters the second stage for
decompressing and verifying. After that it would queue a new item
to copy the original file to the location, but that copy item would
be in the wrong stage, causing it to use the hashes for the
decompressed item.
Closes: #838441
|
|
Config options are checked in various paths, so making "useless" memory
allocations wastes time and can also cause problems like #852757.
The unneeded malloc was added in ae73a2944a89e0d2406a2aab4a4c082e1e9da3f9.
(We have no explicit malloc here – its std:string doing this internally)
|
|
Most of them in (old) code comments. The two instances of user visible
string changes the po files of the manpages are fixed up as well.
Gbp-Dch: Ignore
Reported-By: spellintian
|
|
Nothing in user visible strings.
Gbp-Dch: Ignore
Reported-By: codespell
|
|
If apt renames a file to .FAILED it leaves its namespace and is never
touched again – expect since 1.1~exp4 in which "apt clean" will remove
those files. The usefulness of these files rapidly degrades if you don't
keep the update log itself (together with debug output in the best case)
through and on 99% of all system they will be kept around forever just
to collect dust over time and eat up space.
With this commit an update call will remove all FAILED files of previous
runs, so that the FAILED files you have on disk are always only the ones
related to the last apt run stopping apt from hoarding files.
Closes: 846476
|
|
Keeping the Fd of the cache file we have validated around to later load
it into the mmap ensures not only that we load the same file (which
wouldn't really be a problem in practice), but that this file also still
exists and wasn't deleted e.g. by a 'apt clean' call run in parallel.
|
|
This will avoid people from thinking that they have to do nothing
when they change the set of files.
Gbp-Dch: ignore
|
|
This is somewhat more portable than just hardcoding perl or in the
triehash case /usr/bin/perl in the shebang.
Thanks: Guillem Jover for the hint
Gbp-Dch: ignore
|
|
Our implementation of wildcards was rudimentary. It worked for some
common ones, but it was also broken: For example, armel matched any-armel,
but should match any-arm.
With this commit, we load the correct tables from dpkg. Supported are
both triplets and quadruplet tables (the latter introduced in dpkg 1.18.11).
There are some odd things we have to deal with in the cache filter for
historical and API reasons:
* The character "*" must be accepted as an alternative to any - in fact
it may appear anywhere in the wildcard as we also allow fnmatch() style
wildcard matching on the commandline.
* The code might get passed an arch with a minus at the end, for example
the cmdline "install apt:any-arm-" will first try to check if any-arm-
is a valid architecture. We deal with this by rejecting any wildcard
ending in a minus.
* Triplets are actually implemented by extending them to faux quadruplets
- by prepending a "base" component for the architecture tuple, and "any"
if there is a wildcard component.
Once we have constructed a wildcard, it is transformed into an fnmatch()
expression for historical reasons. In the future, we should really get a
tuple class and implement matching in a better, more explicit way.
This does for now though - it passes all the test cases and accepts all
things it should accept.
Closes: #748936
Thanks: James Clarke <jrtc27@jrtc27.com> for the initial patch
|
|
Thanks: James Clarke <jrtc27@jrtc27.com> for the implementation
Gbp-Dch: ignore
|
|
This is useful for e.g. Britney, where the Build-Depends would have to
be parsed for multiple architectures. With this change, the call can
choose the architecture without having to mess with the config.
Signed-off-by: Niels Thykier <niels@thykier.net>
Closes: #845969
(jak@d.o: made the code compile)
|
|
The idea is simple: Each¹ Find*( call starts with a call check if the
given option (with the requested type) exists in the whitelist. The
whitelist is specified via our configure-index file so that we have
a better chance at keeping it current. the whitelist is loaded via a
special (undocumented for now) configuration stanza and if none is
loaded the empty whitelist will make it so that no warnings are shown.
Much needs to be done still, but that is as good a time as any to take a
snapshot of the current state and release it into the wild given that it
found some bugs already and has no practical effect on users.
¹ not all in this iteration, but many
|
|
Interpreting a boolean as an int works just fine – it just hasn't the
intended result – it isn't a serious problem through as the disabling of
the usage of this dpkg calling style is just an "optimization"
|
|
Again no practical difference, but for consistency a boolean option
should really be accessed via a boolean method rather than an int
especially if you happen to try setting the option to "true" …
Gbp-Dch: Ignore
|
|
This can happen e.g. for file: repositories. There is no inherent
problem with setting such values internally, but its bad style,
forbidden in the manpage and could be annoying in the future.
Gbp-Dch: Ignore
|
|
Unlikely to have any practical effect, but its more consistent to use
the right methods instead of performing it slightly incorrect by hand.
Gbp-Dch: Ignore
|
|
The crude way of preparing a message to be a multiline value failed at
generation valid deb822 in case the error message ended with a new line
like the resolving errors from apt do. apt itself can parse these, but
other tools like grep-dctrl choke on it, so be nice and print valid.
Reported-By: Johannes 'josch' Schauer on IRC
|
|
Any respective parser will do the right thing and grab the last value,
but its better for style to generate that field only once.
Gbp-Dch: Ignore
|
|
Clearsigned files like InRelease, .dsc, .changes and co can potentially
include unsigned or additional messages blocks ignored by gpg in
verification, but a potential source of trouble in our own parsing
attempts – and an unneeded risk as the usecases for the clearsigned
files we deal with do not reasonably include unsigned parts (like emails
or some such).
This commit changes the silent ignoring to warnings for now to get an
impression on how widespread unintended unsigned parts are, but
eventually we want to turn these into hard errors.
|
|
Note: This is a warning about disabling a security feature. It is
supposed to be scary as we are disabling a security feature and we
can't just be silent about it! Downloads really shouldn't happen
any longer as root to decrease the attack surface – but if a warning
causes that much uproar, consider what an error would do…
The old WARNING message:
| W: Can't drop privileges for downloading as file 'foobar' couldn't be
| accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied)
is frequently (incorrectly) considered to be an error message indicating
that the download didn't happen which isn't the case, it was performed,
but without all the security features enabled we could have used if run
from some other place…
The word "unsandboxed" is chosen as the term 'sandbox(ed)' is a common
encounter in feature lists/changelogs and more people are hopefully able
to make the connection to 'security' than it is the case for 'privilege
dropping' which is more correct, but far less known.
Closes: #813786
LP: #1522675
|
|
This is a follow up to the previous issue where we did not check
if getline() returned -1 due to an end of file or due to an error
like memory allocation, treating both as end of file.
Here we ensure that we also handle buffered writes correctly by
flushing the files before checking for any errors in our error
stack.
Buffered writes themselves were introduced in 1.1.9, but the
function was never called with a buffered file from inside
apt until commit 46c4043d741cb2c1d54e7f5bfaa234f1b7580f6c
which was first released with apt 1.2.10. The function is
public, though, so fixing this is a good idea anyway.
Affected: >= 1.1.9
|
|
This fixes a security issue where signatures of the
InRelease files could be circumvented in a man-in-the-middle
attack, giving attackers the ability to serve any packages
they want to a system, in turn giving them root access.
It turns out that getline() may not only return EINVAL
as stated in the documentation - it might also return
in case of an error when allocating memory.
This fix not only adds a check that reading worked
correctly, it also implicitly checks that all writes
worked by reporting any other error that occurred inside
the loop and was logged by apt.
Affected: >= 0.9.8
Reported-By: Jann Horn <jannh@google.com>
Thanks: Jann Horn, Google Project Zero for reporting the issue
LP: #1647467
|
|
In ad9416611ab83f7799f2dcb4bf7f3ef30e9fe6f8 we fall back to asking the
original mirror (e.g. a redirector) if we do not get the expected
result. This works for the indexes, but patches are a different beast
and much simpler. Adding this fallback code here seems like overkill as
they are usually right along their Index file, so actually forward the
relevant settings to the patch items which fixes pdiff support combined
with a redirector and partial mirrors as in such a situation the pdiff
patches would be 404 and the complete index would be downloaded.
|
|
We report warnings from apt-key this way already since
29c590951f812d9e9c4f17706e34f2c3315fb1f6, so reporting errors seems like
a good addition. Most of those errors aren't really from apt-key
through, but from the code setting up and actually calling it which used
to just print to stderr which might or might not intermix them with
(other) progress lines in update calls. Having them as proper error
messages in the system means that the errors are actually collected
later on for the list instead of ending up with our relatively generic
but in those cases bogus hint regarding "is gpgv installed?".
The effective difference is minimal as the errors apply mostly to
systems which have far worse problems than a not as nice looking error
message, which makes this pretty hard to test – but at least now the
hint that your system is broken can be read in proper order (= there
aren't many valid cases in which the permissions of /tmp are messed up…).
LP: #1522988
|
|
|
|
We try to configure all packages at the end which need to be configured,
but that also applies to packages which weren't completely installed
(e.g. maintainerscript failed) we end up removing in this interaction
instead.
APT doesn't perform this explicit configure in the end as it is using
"dpkg --configure --pending", but it does confuse the progress report
and potentially also hook scripts.
Regression-Of: 9ffbac99e52c91182ed8ff8678a994626b194e69
|
|
dpkg stumbles over these (#844300) and we haven't dropped 'easier'
removes to be implicit and to be scheduled by dpkg by default so far
so we shouldn't push the decision in such cases to dpkg either.
|
|
Our old idea was to look for the first package which would be "touched"
and take this as the package dpkg is talking about, but that is
incorrect in complicated situations like a package upgraded to/from
multiple M-A:same siblings installed.
As we us the progress report to decide what is still needed we have to
be reasonabily right about the package dpkg is talking about, so we jump
to quite a few loops to get it.
|
|
Given that we use the progress information to skip over actions dpkg has
already done like not purging a package which was already removed and
had no config files or not acting on disappeared packages and such it is
important that apt and dpkg agree on which states the package has to
pass through.
To ensure that we keep tabs on this in the future a warning is added at
the end if apt hasn't seen all the action it was supposed to see. I
can't wait for the first bugreporters to wonder about this…
|
|
If a package is triggered dpkg frequently issues two messages about it
causing us to make a note about it both times which messes up our
planned dpkg actions view. Adding these actions if we have nothing else
planned fixes this and should still be correct as those planned actions
will deal with the triggering just fine and we avoid strange problems
like a package triggered before its removed…
|
|
Our profile says we spend about 5% of the time transforming the
hex digits into the binary format used by HashsumValue, all for
comparing them against the other strings. That makes no sense
at all.
According to callgrind, this reduces the overall instruction
count from 5,3 billion to 5 billion in my example, which
roughly matches the 5%.
|
|
Generating a string for each version we see is somewhat inefficient.
The problem here is that the Description tag names are longer than
15 byte, and thus require an allocation on the heap, which we should
avoid.
It seems reasonable that 20 characters works for all languages codes
used for archive descriptions, but if not, there's a warning, so
we'll catch that.
This should improve performance by about 2%.
|
|
This has the effect of significantly reducing actual string
comparisons, and should improve the performance of FindGrp
a bit, although it's hardly measureable (callgrind says it
uses 10% instructions less now).
|
