| Age | Commit message (Collapse) | Author |
|---|
|
Reference mail:
https://lists.debian.org/debian-l10n-english/2015/11/msg00006.html
|
|
In 8d041b4f we made apt figure out based on the last Release file it has
if it should request a file or not given that the hashes changed or not.
So if we have a last Release file and do a request, do not sent a
Last-Modified header as we expect a change so much that a non-change
would indeed be an error. The Last-Modified header is therefore at best
ignored by the server, so sending it is just wasted effort. In the worst
case as time is a fragile thing the server decides against sending us an
update with the idea that we already have the latest content, which we
know for a fact that we haven't. Given that we sent less information to
the server our request is on its own also less identifiable as coming
from a returning or new user.
The disadvantage is that if we end up getting an old index file after
getting a new Release file from another mirror the old mirror will not
be able to tell us 'Hit', but instead sends us the complete file we
discard, but both lets us end up with the same error class in the end,
so the difference isn't big in practice.
|
|
In ce1f3a2c we started warning about failing unlinking, which we
consistently do for directories. That isn't a problem as directories
usually aren't in the places we do want to clean up – with the potential
exeception of "lost+found", so lets ignore it like we ignore our own
partial/ subdirectory.
Closes: 805424
|
|
Git-Dch: Ignore
|
|
space-gapping: '-o option= value'
That is a very old feature (straight from 1998), but it is super
surprising if you try setting empty values and instead get error
messages or a non-empty value as the next parameter is treated as the
value – which could have been empty, so if for some reason you need a
compatible way of setting an empty value try: '-o option="" ""'.
I can only guess that the idea was to support '-o option value', but we
survived 17 years without it, we will do fine in the future I guess.
Similar is the case for '-t= testing' even through '-t testing' existed
before and the code even tried to detect mistakes like '-t= -b' … all
gone now.
Technically that is as its removing a feature replacing it with another
a major interface break. In practice I really hope for my and their
sanity that nobody was using this; but if for some reaon you do: Remove
the space and be done.
I found the patch and the bugreport actually only after the fact, but
its reassuring that others are puzzled by this as well and hence a
thanks is in perfect order here as the patch is practical identical
[expect that this one here adds tests and other bonus items].
Thanks: Daniel Hartwig for initial patch.
Closes: 693092
|
|
In 0940230d we started dropping privileges for file (and a bit later for
copy, too) with the intend of uniforming this for all methods. The
commit message says that the source will likely fail based on the
compressors already – and there isn't much secret in the repository
content. After all, after apt has run the update everyone can access the
content via apt anyway…
There are sources through which worked before which are mostly
single-deb (and those with the uncompressed files available).
The first one being especially surprising for users maybe, so instead of
failing, we make it so that apt detects that it can't access a source as
_apt and if so doesn't drop (for all sources!) privileges – but we limit
this to file/copy, so the uncompress which might be needed will still
fail – but that failed before this regression.
We display a notice about this, mostly so that if it still fails (e.g.
compressed) the user has some idea what is wrong.
Closes: 805069
|
|
Notices are just hints, but if they are printed in tests, they should be
expected and if not the test should fail. No current test has this
problem, so that is just potential future proving.
Git-Dch: Ignore
|
|
This 'ignores' the component Release files you can find in Debian
alongside the binary-* directories, which isn't exactly a common
usecase, but it worked before, so lets support it again as this isn't
worse than a valid Release file which is unsigned.
Git-Dch: Ignore
|
|
Allows users who know what they are getting themselves into with this
trick to e.g. disable privilege dropping for e.g. file:// until they can
fix up the permissions on those repositories. It helps also the test
framework and people with a similar setup (= me) to run in less modified
environments.
|
|
Continueing on the track of dropping privileges in all methods, lets
drop it in copy, too, as the reasoning for it is very similar to file
and the interaction between the too quiet interesting as copy kinda
surfed as a fallback for file not being able to read the file. Both now
show a better error message as well as it was previously claiming to
have a hashsum mismatch, given that it couldn't read the file.
Git-Dch: Ignore
|
|
This flags is generally handy to avoid having to deal with ipv6 results on an
ipv4-only system, but it prevents e.g. the testcases from working if the
testsystem has no configured address at the moment (expect loopback), so
allow it to be sidestepped and let the testcases sidestep it.
Git-Dch: Ignore
|
|
The wrapping will fail in the best case and actually end up deleting
/dev/null in the worst case. Given that there is no point in trying to
write atomically to /dev/null as you can't read from it again just
ignore these flags if higher level code ends up trying to use them on
/dev/null.
Git-Dch: Ignore
|
|
Leading or trailing newlines can be confusing for our parser as it
expects two newlines to start/stop a new stanza. To solve this the lines
we wanna add are printed first, ignore any leading newlines and then add
the stanza as provided by dpkg-deb with or without trailing newlines as
the parser will look at the first stanza only anyway and removing
trailing newlines is considerably harder to do.
Closes: 802553
|
|
Based on a discussion with Niels Thykier who asked for Contents-all this
implements apt trying for all architecture dependent files to get a file
for the architecture all, which is treated internally now as an official
architecture which is always around (like native). This way arch:all
data can be shared instead of duplicated for each architecture requiring
the user to download the same information again and again.
There is one problem however: In Debian there is already a binary-all/
Packages file, but the binary-any files still include arch:all packages,
so that downloading this file now would be a waste of time, bandwidth
and diskspace. We therefore need a way to decide if it makes sense to
download the all file for Packages in Debian or not. The obvious answer
would be a special flag in the Release file indicating this, which would
need to default to 'no' and every reasonable repository would override
it to 'yes' in a few years time, but the flag would be there "forever".
Looking closer at a Release file we see the field "Architectures", which
doesn't include 'all' at the moment. With the idea outlined above that
'all' is a "proper" architecture now, we interpret this field as being
authoritative in declaring which architectures are supported by this
repository. If it says 'all', apt will try to get all, if not it will be
skipped. This gives us another interesting feature: If I configure a
source to download armel and mips, but it declares it supports only
armel apt will now print a notice saying as much. Previously this was a
very cryptic failure. If on the other hand the repository supports mips,
too, but for some reason doesn't ship mips packages at the moment, this
'missing' file is silently ignored (= that is the same as the repository
including an empty file).
The Architectures field isn't mandatory through, so if it isn't there,
we assume that every architecture is supported by this repository, which
skips the arch:all if not listed in the release file.
|
|
This was discussed a while ago on #debian-apt and now that I see myself
making this mistake lets bite the bullet and fix it in the easy way out
version: Using a new name which fits with a similar named setter and
deprecate the old method instead of 'hostily' changing API.
Closes: #803471
|
|
-q is for logging and -qqq (old -qq) basically kills every output expect
errors, so there should be a way of declaring a middleground in which
the output of e.g. 'update' isn't as verbose, but still shows some
things. The test framework was actually making use of by accident as it
ignored the quiet level in output setup for apt before.
Eventually we should figure out some better quiet levels for all tools…
|
|
apt is an interactive command and the reasons we haven't this option set
for everything is mostly in keeping compatibility for a little while
longer to allow scripts to be changed if need be.
|
|
Git-Dch: Ignore
|
|
Especially with apt now, it can be useful to set an option only for apt
and not for apt-get. Using a binary-specific subtree which is merged into
the root seems like a simple enough trick to achieve this.
|
|
The bugreport is more conservative in asking for a conditional, but
given that this is a message intended to be read by users to be run by
users we should suggest using a command intended to be used by users.
And while we are at, add sudo to the message – conditional of course.
Closes: 801571
|
|
The manpage is also slightly updated to work better as a central hub to
push people from all angles into the right directions without writting a
book disguised as an error message.
|
|
Insecure (aka unsigned) repositories are bad, period. We want to get
right of them finally and as a first step we are printing scary
warnings. This is already done, this commit just changes the messages to
be more consistent and prevents them from being displayed if
authenticity is guaranteed some other way (as indicated with
trusted=yes).
The idea is to first print the pure fact like "repository isn't signed"
as a warning (and later as an error), while giving an explaination in a
immediately following notice (which is displayed only in quiet level 0:
so in interactive use, not in scripts and alike).
Closes: 796549
|
|
The main part is refactoring through to allow hiding the magic needed to
support .deb files in deeper layers of libapt so that frontends have
less exposure to Debian specific classes like debDebPkgFileIndex.
|
|
Commit 653ef26c70dc9c0e2cbfdd4e79117876bb63e87d broke the camels back in
sofar that everything works in terms of our internal use of copy:/, but
external use is completely destroyed. This is kinda the reverse of what
happened in "parallel" in the sid branch, where external use was mostly
fine, internal and external exploded on the GzipIndexes option.
We fix this now by rewriting our internal use by letting copy:/ only do
what the name suggests it does: Copy files and not uncompress them
on-the-fly. Then we teach copy and the uncompressors how to deal with
/dev/null and use it as destination file in case we don't want to store
the uncompressed files on disk.
Closes: 799158
|
|
Git-Dch: Ignore
|
|
dpkg-checkbuilddeps changed its exitcodes in the recent past so the
old check always fails now skipping the test. Lets try a slightly more
stable (at least assume it to be) variant of detecting this.
See also 4f6d26b4d41474aa390329b7e9cb167eb70b2821.
Git-Dch: Ignore
|
|
As we have support for 'hold', we need support for undoing a hold which
in effect means that we implemented most other states as well, just that
they weren't exposed in the interface directly so far.
|
|
We had this code lying around in apt-mark for a while now, but other
frontends need this (and similar) functionality as well, so its high
time that we provide a public interface in libapt for this stuff.
|
|
Since gcc 4.9, the API for erase slightly changed. In
commit 3dddcdf2432e78f37c74d8c76c2c519a8d935ab2 the
existing checks for __cplusplus where changed to
check the gcc version, as the __cplusplus check
did nothing, because gcc 4.8 already provided the
standard value in there.
Fix the code to check for the gcc version in two
more places, and change the existing checks to
use a convenience macro.
|
|
Now that tests can be run in parallel, lets actually do it… The mode has
some downsides like not collecting the failed tests, but it can be a lot
faster than a sequential run and is therefore a good alternative in
testing those "this shouldn't break anything" changes (which tend to
break everything if untested).
Git-Dch: Ignore
|
|
This allows running tests in parallel.
Git-Dch: Ignore
|
|
We uses a small trick to implement the fallback: We make it so, that
by-hash is a special compression algorithm and apt already knows how to
deal with fallback between compression algorithms.
The drawback with implementing this fallback is that a) we are guessing
again and more importantly b) by-hash is only tried for the first
compression algorithm we want to acquire, not for all as before – but
flipping between by-hash and well-known for each compression algorithm
seems to be not really worth it as it seems unlikely that there will
actually be mirrors who only mirror a subset of compressioned files, but
have by-hash enabled.
The user-experience is the usual fallback one: You see "Ign" lines in
the apt update output. The fallback is implemented as a transition
feature, so a (potentially huge) mirror network doesn't need a flagday.
It is not meant as a "someday we might" or "we don't, but some of our
mirrors might" option – we want to cut down on the 'Ign' lines front so
that they become meaningful – if we wanted to spam everyone with them, we
could enable by-hash by default for all repositories…
sources.list and config options are better suited for this.
Closes: 798919
|
|
This changes the semantics of the option (which is renamed too) to be a
yes/no value with the special additional value "force" as this allows
by-hash to be disabled even if the repository indicates it would be
supported and is more in line with our other yes/no options like pdiff
which disable themselves if no support can be detected.
The feature wasn't documented so far and hasn't reached a (un)stable
release yet, so changing it without trying too hard to keep
compatibility seems okay.
|
|
Not all tests work yet, most notable the cdrom tests, but those require
changes in libapt itself to have a proper fix and what we have fixed so
far is good enough progress for now.
Git-Dch: Ignore
|
|
Filenames we get could include spaces, but also the tmpdir we work in
and the failures we print in return a very generic and unhelpful…
Properly supporting spaces is a bit painful as we constructed gpg
command before, which is now moved to (multilevel) calls to temporary
scripts instead.
|
|
This is mostly a small speedup for the testcases, but it is also handy
to document which tests actually deal with a specific hash compared to
those which 'just' need some hash which can be important while adding
new hashes.
Git-Dch: Ignore
|
|
Reported-By: Konomi on IRC
|
|
And of course, testing obscure things ends up showing obscure 'bugs' or
better shortcomings/inconsitencies, so lets fix them with the tests.
Git-Dch: Ignore
|
|
A bit unfair that only Bzr had this message. Lets at least print it for
git as well with the option of adding more later without string changes.
|
|
The tests usually run on amd64 boxes, but once in a while I run it on a
(slow) armel box as well, which has its fair share of problems with some
tests, but at least the low hanging fruits can be dealt with: Do not
assume that amd64 is the native dpkg architecture – instead use whatever
dpkg thinks is native as architecture for the test.
Git-Dch: Ignore
|
|
Our error reporting is historically grown into some kind of mess.
A while ago I implemented stacking for the global error which is used in
this commit now to wrap calls to functions which do not report (all)
errors via return, so that only failures in those calls cause a failure
to propergate down the chain rather than failing if anything
(potentially totally unrelated) has failed at some point in the past.
This way we can avoid stopping the entire acquire process just because a
single source produced an error for example. It also means that after
the acquire process the cache is generated – even if the acquire
process had failures – as we still have the old good data around we can and
should generate a cache for (again).
There are probably more instances of this hiding, but all these looked
like the easiest to work with and fix with reasonable (aka net-positive)
effects.
|
|
In private-install.cc we call MarkInstall with FromUser=true, which sets
the bit accordingly, but while applying the EDSP solution we call mark
install on all packages with FromUser=false, so MarkInstall believes
this install is an automatic one and sets it to auto – so that a new package
which is explicitely installed via an external solver is marked as auto
and is hence also up for garbage collection in a following call.
Ideally MarkInstall wouldn't reset it, but the detection is hard to do
without regressing in other cases – and ideally ideally MarkInstall
wouldn't deal with the autobit at all – so we work around this on the
calling side for now.
|
|
The parser creates a preferences as well as an extended states file
based on the EDSP scenario file, which isn't the most efficient way of
dealing with this as thes text files have to be parsed again by another
layer of the code, but it needs the least changes and works good enough
for now. The 'apt' solver is in the end just a test solver like dump.
|
|
This is basically a rewrite of the script with the general idea of
finding the Debian version of the installed kernels – as multiple
flavours will have the same Debian version – select the two newest of
them and translate them back to versions found in package names.
This way we avoid e.g. kernel and kernel-rt to use up the protected
slots even through they are basically the same kernel (just a different
flavour) so it is likely that if kernel doesn't work for some reason,
kernel-rt will not either.
This also deals with foreign kernel packages, kernels on hold and partly
installed kernels (in case multiple kernels are installed in the same
apt run) in a hopefully sensible way.
Closes: 787827
|
|
As said in the bugreport, this is hardly a serious problem on a security
front, but it was always on the list to have the filename configurable
somehow and the stable filename is a problem for parallel executions.
Using an environment variable (APT_EDSP_DUMP_FILENAME) for this is more
or less the best we can do here as solvers do not get told about our
configuration and such.
Closes: 795600
|
|
The syntax of "Source" is different in EDSP compared to the the field of
the same name in 'the rest' of Debian, so documented this accordingly
and send the version as a new field.
|
|
How the Multi-Arch field and pkg:<arch> dependencies interact was
discussed at DebConf15 in the "MultiArch BoF". dpkg and apt (among other
tools like dose) had a different interpretation in certain scenarios
which we resolved by agreeing on dpkg view – and this commit realizes
this agreement in code.
As was the case so far libapt sticks to the idea of trying to hide
MultiArch as much as possible from individual frontends and instead
translates it to good old SingleArch. There are certainly situations
which can be improved in frontends if they know that MultiArch is upon
them, but these are improvements – not necessary changes needed
to unbreak a frontend.
The implementation idea is simple: If we parse a dependency on foo:amd64
the dependency is formed on a package 'foo:amd64' of arch 'any'. This
package is provided by package 'foo' of arch 'amd64', but not by 'foo'
of arch 'i386'. Both of those foo packages provide each other through
(assuming foo is M-A:foreign) to allow a dependency on 'foo' to be
satisfied by either foo of amd64 or i386. Packages can also declare to
provide 'foo:amd64' which is translated to providing 'foo:amd64:any' as
well.
This indirection over provides was chosen as the alternative would be to
teach dependency resolvers how to deal with architecture specific
dependencies – which violates the design idea of avoiding resolver
changes, especially as architecture-specific dependencies are a
cornercase with quite a few subtil rules. Handling it all over versioned
provides as we already did for M-A in general seems much simpler as it
just works for them.
This switch to :any has actually a "surprising" benefit as well: Even
frontends showing a package name via .Name() [which doesn't show the
architecture] will display the "architecture" for dependencies in which
it was explicitely requested, while we will not show the 'strange' :any
arch in FullName(true) [= pretty-print] either. Before you had to
specialcase these and by default you wouldn't get these details shown.
The only identifiable disadvantage is that this complicates error
reporting and handling. apt-get's ShowBroken has existing problems with
virtual packages [it just shows the name without any reason], so that
has to be worked on eventually. The other case is that detecting if a
package is completely unknown or if it was at least referenced somewhere
needs to acount for this "split" – not that it makes a practical
difference which error is shown… but its one of the improvements
possible.
|
|
Git-Dch: Ignore
|
|
We parse all architectures we encounter recently, which means we also
parse packages from architectures which are neither native nor foreign,
but still came onto the system somehow (usually via heavy force).
|
|
Previously we had python:any:amd64, python:any:i386, … in the cache and
the dependencies of an amd64 package would be on python:any:amd64, of an
i386 on python:any:i386 and so on. That seems like a relatively
pointless endeavor given that they will all be provided by the same
packages and therefore also a waste of space.
Git-Dch: Ignore
|
