| Age | Commit message (Collapse) | Author |
|---|
|
There is a good chance that the attempt will fail, but if a user
mentions certain packages explicitly on the commandline there is a
chance that this will consist of a broken system which is resolved
by upgrading more packages then just the mentioned.
This limitation was not effecting external resolvers.
|
|
With the previous commit we track the state of transactions, so we can
now use our knowledge to avoid processing data for a transaction which
was already closed (via an abort in this case).
This is needed as multiple independent processes are interacting in the
process, so there isn't a simple immediate full-engine stop and it would
also be bad to teach each and every item how to check if its manager
has failed subordinate and what to do in that case.
In the pdiff case, which deals (potentially) with many items during its
lifetime e.g. a hashsum mismatch in another file can abort the
transaction the file we try to patch via pdiff belongs to. This causes
some of the items (which are already done) to be aborted with it, but
items still in the process of acquisition continue in the processing and
will later try to use all the items together failing in strange ways as
cleanup already happened.
The chosen solution is to dry up the communication channels instead by
ignoring new requests for data acquisition, canceling requests which are
not assigned to a queue and not calling Done/Failed on items anymore.
This means that e.g. already started or pending (e.g. pipelined)
downloads aren't stopped and continue as normal for now, but they remain
in partial/ and aren't processed further so the next update command will
pick them up and put them to good use while the current process fails
updating (for this transaction group) in an orderly fashion.
Closes: 817240
Thanks: Barr Detwix & Vincent Lefevre for log files
|
|
We want to keep track of the state of a transaction overall to base
future decisions on it, but as a pre-requirement we have to make sure
that a transaction isn't commited twice (which happened if the download
of InRelease failed and Release takes over).
It also happened to create empty commits after a transaction was already
aborted in cases in which the Release files were rejected.
This isn't effecting security at the moment, but to ensure this isn't
happening again and can never be bad a bunch of fatal error messages are
added to make regressions on this front visible.
|
|
Git-Dch: Ignore
Reported-By: gcc -fsanitize=address
|
|
Hardly noticeable, but given that we have the option to easily enable
it, lets enable it as every newline in the message is written
individually by the code.
|
|
Some methods had it missing, some used the keyword directly, which isn't
a problem as it is a cc file, but for consistency lets stick to our
macro for now.
Git-Dch: Ignore
|
|
Introduces APT::Hashes::<NAME> with entries Untrusted and Weak
which can be set to true to cause the hash to be treated as
untrusted and/or weak.
|
|
If the package is marked for removal, keep it marked for
removal and do not mark it for keep. If we mark it for keep,
we some how later get to a different stage where it is marked
for unpack instead of removal.
In the example in the bug report, we would get a:
SmartUnPack maas-region-controller-min:amd64 (replace version 2.0.0~alpha3+bzr4810-0ubuntu1 with Segmentation fault
maas-region-controller-min:amd64 was marked for removal, but
we changed it to keep and somehow it thinks that this is to
be replaced now instead of removed (probably because the
InstallVer != CandidateVer [with InstallVer = 0]).
This fixes a regression introduced in release 1.2.7, commit:
0390edd5452b081f8efcf412f96d535a1d959457
Reported-by: LaMont Jones on IRC
LP: #1562402
|
|
|
|
This avoids templates using StringView to be exported, such as
std::vector<StringView*>::emplace_back().
Gbp-Dch: ignore
|
|
There is really no need to have the same code three times.
Git-Dch: Ignore
|
|
Otherwise, things will just start failing later down the stack,
because (a) the lazy getters do not check if building was successful
and (b) any further getter call would return the invalid object
anyway.
Also initialize VS in pkgCache to nullptr by default.
Closes: #818628
|
|
The epoch stripping in this code is done since day one, but in other
places we show a version epochs are not stripped. If epochs are present
in packages they tend to be an important information which we can't just
drop and especially can't drop "sometimes" as that confuses users and
tools alike – so even if removing code in use for (close to) 18 years
feels wrong, it is probably the right choice for consistency.
Closes: 818162
|
|
This makes the new GPG related warnings much nicer to read,
for example, the second one here replaces the first one:
W: gpgv:/var/lib/apt/lists/example.com_dists_stable_InRelease: Weak ...
W: http://example.com/dists/stable/InRelease: Weak ...
|
|
This makes it easier to understand what really is an error
and what not.
|
|
For the non-pdiff case, we have can have accurate progress
reporting because after fetching the {,In}Release files we know
how many IndexFiles will be fetched and what size they have.
Therefore init the filesize early (in pkgAcqIndex::Init) and
ensure that in Acquire::Pulse() looks at already downloaded
bits when calculating the progress in Acquire::Pulse.
Also improve debug output of Debug::acquire::progress
|
|
The problemresolver will set the candidate version for pkg P back
to the current version if it encounters an impossible to satisfy
critical dependency on P. However it did not set the State of
the package back as well which lead to a situation where P is
neither in Keep,Install,Upgrade,Delete state.
Note that this can not be tested via the traditional sh based
framework. I added a python-apt based test for this.
LP: #1550741
[jak@debian.org: Make the test not fail if apt_pkg cannot be
imported]
|
|
This can be used by workers to send warnings to the main
program. The messages will be passed to _error->Warning()
by APT with the URI prepended.
We are not going to make that really public now, as the
interface might change a bit.
|
|
The structure we parse the data into has a dedicated size field, but it
tends to be easier to handle it as a (very weak) checksum.
|
|
The URI descibing an item can change via mirrors/redirectors which
causes the .diff/Index files to get the wrong names in storage.
Git-Dch: Ignore
|
|
The (unlikely) waitpid failure case should fallthrough the code just
like the other failures (and successes) instead of taking a shortcut
avoiding all the cleanup (progress) and finishing touches (log, state).
This also delays the cleanup of the progress until apt is really done
with everything and "just" has the post-invokes left to do, so the
period of 'apt looks finished as it stopped the progress' and 'apt
really finished as I have the shell-prompt back' is shorter even if
there is no progress reported anymore, so the bar lingers at 100%…
Ideally even the post-invokes would be covered by progress, but they
can have their own output and dealing with that could be hard.
Git-Dch: Ignore
|
|
All other interactions with std::cout are flushed directly, just in the
stop case we hadn't done it – no problem expect if there is still output
coming after apt is done like in the case of a post-invoke script
producing output.
Closes: 793672
|
|
Now that we ignore SHA1-only files it makes sense to require also the
provision of hashes for the compressed patches as this was introduced in
the same patchset as support for non-SHA1 hashes in the file itself in
dak and adding support in other archive creators (if they support pdiffs
at all) will likely be in the same batch.
The reason for the change itself is simple: If you are 'scared' enough
about the security of SHA1, you shouldn't uncompress a file you haven't
verified at all – after all, it could be exploiting a bug or a zip bomb.
|
|
SHA1 is not reasonably secure anymore, so we should not consider it
usable anymore. The test suite is adjusted to account for this.
|
|
Dynamically allocate KillList in order to avoid an overflow when
more than 100 elements would be written to it.
This happened while playing around with the status file from
Bug#701069 on a modern system.
|
|
This effectively merges branch 'typofixes-vlajos-20150807' of github.com:vlajos/apt
with the following commit:
commit 13cacb3e2e2352ba701e769fc889e3344fabbf7e
Author: Veres Lajos <vlajos@gmail.com>
Date: Sun Aug 9 00:12:53 2015 +0100
typofix - https://github.com/vlajos/misspell_fixer
It has been rebased for a better commit message.
|
|
Mysteriously segfaults only on i386 for me, but at least one reporter
had the same behavior and it makes sense that this is the problem as the
parsing of Source: was fixed in 1.2.2 – before the not remapped group
was not used.
We don't use our usual Dynamic<> trick here as we don't have it in the
parser. Its a bit of a layer violation to do this parsing here, but its
how it is always was…
Until next time with this lovely kind of problem.
Closes: 812251
Thanks: Francesco Poli and Marc Haber for testdata.
|
|
If an iterator or a stringview has multiple dynamic objects
registered with it, it may be remapped twice. Prevent that
by noting which iterators/views we have seen and not remapping
one if we have already seen it.
We most likely do not have any instance of multiple dynamics
on a single object, but let's play safe - the overhead is not
high.
|
|
If a single pdiff fails, we have to fail the entire patching endeavour
and fall back to getting the complete file instead. That is easy in
serverside merged pdiffs as we get them one by one. For clientside we
get them all at once through, which means that a failure in one has to
stop the entire pipeline, which works as expected (as proven by the
bugreporters as they don't even notice it happening). The problem is
just that the first failing pdiff will do the cleanup, so another pdiff
which happens to be successfully acquired after we processed the failure
doesn't find the file it is supposed to use as a basename anymore, so
the patch is renamed to what should be the unique extension and moved
into the current working directory. Processing is then stopped as the
patch realizes that it isn't the last one which completed downloading.
On the plus side this means this is neither us using a bad temporary
location nor a security problem. It "just" overrides unconditionally
files in your current working directory (if you happen to have them
named like a pdiff patch – a bit unlikely perhaps) and so drops files
there which are never used again.
I guess this was introduced in 4e3c5633b1e74b4f58b95f339cfbbf4cbf21ab3e
for real as I made the need for the existence of the base file rather
explicit, but the potential lingers in the code for far longer.
Closes: #816837
|
|
The liblzma-based write code needs the same tweaks that the read code
already has to cope with the situation where lzma_code returns zero the
first time through because avail_out is zero, but will do more work if
called again.
This ports the read tweaks to the write code as closely as possible
(including matching comments etc.).
Closes: #751688
|
|
Changelogs are relatively small and we have no hashes for them, but we
had partial support for them before, so lets stick to it.
This also deletes the (partial) file before moving the downloaded file
into its place – rename(2) should be doing this by itself, but testing
on semaphoreci suggests that this isn't always the case (error is "Stale
file handle") and we don't need an atomic replace here, so be explicit.
Git-Dch: Ignore
|
|
Reported-By: Helmut Grohne on IRC
|
|
If the architecture list is empty somehow, fail normally.
LP: #1549819
|
|
The EDSP output generated by apt didn't include the versioned provides
information so that every provides looked like an unversioned one in the
eyes of an external resolver.
|
|
pkgAcqChangelog has the default behaviour of downloading a changelog to
a temporary directory (inside /tmp, not /tmp directly), which is cleaned
up on shutdown, but this can be overridden to store the changelog more
permanently – but that caries a permission problem.
For changelog we can 'easily' solve this by always downloading to a
temporary directory and only move it out of there on done.
|
|
If pkgAcqChangelog is told to acquire the changelog for a version it
will check first if this version is installed on the disk and if so will
use the local changelog in /usr/share/doc (possibily/likely gz
compressed) instead of downloading the file from the web.
An option is provided to disable this, which is enabled by default for
the Ubuntu vendor as they truncate the local changelogs – and for apts
--print-uris action.
|
|
Regression introduced in a249b3e6fd798935a02b769149c9791a6fa6ef16, which
in the case of an invalid cache would build the first part unlocked and
later pick up the (still unlocked) cache for further processing, so the
system got never locked and apt would end up complaining about being
unable to release the lock at shutdown.
The far more common case of having a valid cache worked as expected and
hence covered up the problem – especially as tests who would have
noticed it are simulations only, which do not lock.
Closes: 814139
Reported-By: Balint Reczey <balint@balintreczey.hu>
Reported-By: Helmut Grohne <helmut@subdivi.de> on IRC
|
|
If we just reopened the file, we also need to reset the current
seek position when we reset the buffer, otherwise the code will
not try to seek to the position given to Skip (from 0), but will
try to seek to old offset + the position given to skip.
Closes: #812994, #813000
|
|
When writing into the buffer write to free() bytes starting
at getend(), instead of buffersize_max bytes at get()
-> get() is a read pointer.
This makes no difference in practice though, as we reset
the buffer before the call, so start = end = 0.
Gbp-Dch: ignore
|
|
We cannot just return false without setting an error,
as InternalWrite does not set one itself.
|
|
|
|
Include <stdlib.h> to ensure that mkdtemp(3) is defined to improve
general portability and fix a specific build failure on Android.
Closes: #807367
|
|
Microoptimization, but still gives a measurable 2-3% improvement
when using commands with lots of output like `apt list`.
|
|
It makes no sense to split a large block into multiple small
blocks, so when we have the chance to write them unbuffered,
do so.
|
|
We do not need the loop, FileFd::Private() handles this for us.
Gbp-Dch: ignore
|
|
We want to check whether the amount of free space is smaller
than the requested write size. Checking maxsize - size() is
incorrect for bufferstart >= 0, as size() = end - start.
Gbp-Dch: ignore
|
|
APT has a different understanding than dpkg (#748936) what matches and
what doesn't match an architecture specification as it isn't converting
back (and forward) to Debian triplets. That has to eventually be solved
some way or the other, but until that happens we change the matching in
apt so that porters can continue their work on non-gnu libc-ports even
if policy doesn't specify that yet (and dpkg just supporting it "by
accident" via triplets).
The initial patch was reformatted, fixed in terms of patterns containing
"any-any", dealing with expanding an arch without libc to gnu while a
pattern expands libc to any, the parsedepends test was fixed (the new
if's were inserted one step too early) and another test just for the
specifications added.
Closes: #812212
Thanks: Bálint Réczey for initial patch
|
|
Git-Dch: Ignore
|
|
|
|
The Date field in the Release file is useful to avoid allowing an
attacker to 'downgrade' a user to earlier Release files (and hence to
older states of the archieve with open security bugs). It is also needed
to allow a user to define min/max values for the validation of a Release
file (with or without the Release file providing a Valid-Until field).
APT wasn't formally requiring this field before through and (agrueable
not binding and still incomplete) online documentation declares it
optional (until now), so we downgrade the error to a warning for now to
give repository creators a bit more time to adapt – the bigger ones
should have a Date field for years already, so the effected group should
be small in any case.
It should be noted that earlier apt versions had this as an error
already, but only showed it if a Valid-Until field was present (or the
user tried to used the configuration items for min/max valid-until).
Closes: 809329
|
