| Age | Commit message (Collapse) | Author |
|---|
|
How the Multi-Arch field and pkg:<arch> dependencies interact was
discussed at DebConf15 in the "MultiArch BoF". dpkg and apt (among other
tools like dose) had a different interpretation in certain scenarios
which we resolved by agreeing on dpkg view – and this commit realizes
this agreement in code.
As was the case so far libapt sticks to the idea of trying to hide
MultiArch as much as possible from individual frontends and instead
translates it to good old SingleArch. There are certainly situations
which can be improved in frontends if they know that MultiArch is upon
them, but these are improvements – not necessary changes needed
to unbreak a frontend.
The implementation idea is simple: If we parse a dependency on foo:amd64
the dependency is formed on a package 'foo:amd64' of arch 'any'. This
package is provided by package 'foo' of arch 'amd64', but not by 'foo'
of arch 'i386'. Both of those foo packages provide each other through
(assuming foo is M-A:foreign) to allow a dependency on 'foo' to be
satisfied by either foo of amd64 or i386. Packages can also declare to
provide 'foo:amd64' which is translated to providing 'foo:amd64:any' as
well.
This indirection over provides was chosen as the alternative would be to
teach dependency resolvers how to deal with architecture specific
dependencies – which violates the design idea of avoiding resolver
changes, especially as architecture-specific dependencies are a
cornercase with quite a few subtil rules. Handling it all over versioned
provides as we already did for M-A in general seems much simpler as it
just works for them.
This switch to :any has actually a "surprising" benefit as well: Even
frontends showing a package name via .Name() [which doesn't show the
architecture] will display the "architecture" for dependencies in which
it was explicitely requested, while we will not show the 'strange' :any
arch in FullName(true) [= pretty-print] either. Before you had to
specialcase these and by default you wouldn't get these details shown.
The only identifiable disadvantage is that this complicates error
reporting and handling. apt-get's ShowBroken has existing problems with
virtual packages [it just shows the name without any reason], so that
has to be worked on eventually. The other case is that detecting if a
package is completely unknown or if it was at least referenced somewhere
needs to acount for this "split" – not that it makes a practical
difference which error is shown… but its one of the improvements
possible.
|
|
Initializing a random number generator with the time since epoch could
be good enough, but reaches its limits in test code as the 100
iterations might very well happen in the same second and hence the seed
number is always the same… clock() has a way lower resolution so it
changes more often and not unimportant: If many users start the update
at the same time it isn't to unlikely the SRV record will be ordered in
the same second choosing the same for them all, but it seems less likely
that the exact same clock() time has passed for them.
And if I have to touch this, lets change a few other things as well to
make me and/or compilers a bit happier (clang complained about the usage
of a GNU extension in the testcase for example).
|
|
We had a very similar method previously for our own private usage, but
with some generalisation we can move this check into the acquire system
proper so that all frontends profit from this compatibility change.
As we are disabling a security feature here a warning is issued and
frontends are advised to consider reworking their download logic if
possible.
Note that this is implemented as an all or nothing situation: We can't
just (not) drop privileges for a subset of the files in a fetcher, so in
case you have to download some files with and some without you need to
use two fetchers.
|
|
Some targets like Contents-udeb are special-needs targets. Shipping the
configuration snippet for them is okay, but they shouldn't be downloaded
by default. Forcing the user to enable targets by uncommenting targets
is wrong and this would still not really solve the problem completely as
even if you want to download some -udebs it will probably not be for all
sources you have enabled, so having the possibility of disabling a
target by default, but giving the user the option to enable it on a
per-source entry basis is better.
|
|
Some additional files like 'Contents' are very big and should therefore
kept compressed on the disk, which apt-file did in the past. It also
implemented pdiff patching of these files by un- and recompressing these
files on-the-fly, with this commit we can do the same – but we can do
this in both pdiff patching styles (client and server merging) and
secured by hashes.
Hashes are in so far slightly complicated as we can't compare the hashes
of the compressed files as we might compress them differently than the
server would (different compressor versions, options, …), so we must
compare the hashes of the uncompressed content.
While this commit has changes in public headers, the classes it changes
are marked as hidden, so nobody can use them directly, which means the
ABI break is internal only.
|
|
Disabling pdiffs can be useful occasionally, like if you have a fast
local mirror where the download doesn't matter, but still want to use it
for non-local mirrors. Also, some users might prefer it to only use it
for very big indextargets like Contents.
|
|
First of, the temporary directory we download the changelog to needs to
be owned by _apt, but that also means that we don't need to check if we
could/should drop privs as the download happens to a dedicated tempdir
and only after that it is moved to its final location by a privileged user.
|
|
Reported-By: codespell
|
|
|
|
Thanks: Steve Slangasek for the suggestion
Closes: 695633
|
|
|
|
|
|
feature/srv-records
|
|
Conflicts:
cmdline/apt-helper.cc
cmdline/makefile
|
|
This just changes the DoSearch code to use DisplayRecord to
display the record when the full record is requested.
Closes: #660851
[jak@debian.org: Wrote the commit message]
|
|
Simply overriding the value caused patterns that previously matched
a real package name to not match anymore.
Closes: #760868
|
|
Closes: #283400
|
|
|
|
Some people type them instead of autoremove and autoclean, so make
them happy.
Closes: #274159
Makes-Happy: Ansgar
|
|
Closes: #153161
|
|
debian/experimental
|
|
More standardization
|
|
Switch to std::unique_ptr, as this is safer than SPtr.
|
|
Gbp-Dch: ignore
|
|
Also optionally enable old output by setting APT::Policy=0.
|
|
Thanks: Andre Felipe Machado for initial patch
Closes: 414848
|
|
It's gone.
Gbp-Dch: ignore
|
|
Git-Dch: Ignore
|
|
Trade deduplication of code for a bunch of new virtuals, so it is
actually visible how the different indexes behave cleaning up the
interface at large in the process.
Git-Dch: Ignore
|
|
Expecting the worst is easy to code, but has its disadvantages e.g.
by creating package structures which otherwise would have never
existed. By creating the provides instead at the time a package
structure is added we are well prepared for the introduction of partial
architectures, massive amounts of M-A:foreign (and :allowed) and co as
far as provides are concerned at least. We have something relatively
similar for dependencies already.
Many tests are added for both M-A states and the code cleaned to
properly support implicit provides for foreign architectures and
architectures we 'just' happen to parse.
Git-Dch: Ignore
|
|
Before MultiArch implicits weren't a thing, so they were hidden by
default by definition. Adding them for MultiArch solved many problems,
but having no reliable way of detecting which dependency (and provides)
is implicit or not causes problems everytime we want to output
dependencies without confusing our observers with unneeded
implementation details.
The really notworthy point here is actually that we keep now a better
record of how a dependency came to be so that we can later reason about
it more easily, but that is hidden so deep down in the library internals
that change is more the problems it solves than the change itself.
|
|
We aren't and we will not be really compatible again with the previous
stable abi, so lets drop these markers (which never made it into a
released version) for good as they have outlived their intend already.
Git-Dch: Ignore
|
|
Having dependency data separated from the link between version/package
and the dependency allows use to work on sharing the depdency data a bit
as it turns out that many dependencies are in fact duplicates. How many
are duplicates various heavily with the sources configured, but for a
single Debian release the ballpark is 2 duplicates for each dependency
already (e.g. libc6 counts 18410 dependencies, but only 45 unique). Add
more releases and the duplicates count only rises to get ~6 for 3
releases. For each architecture a user has configured which given the
shear number of dependencies amounts to MBs of duplication.
We can cut down on this number, but pay a heavy price for it: In my
many releases(3) + architectures(3) test we have a 10% (~ 0.5 sec)
increase in cache creationtime, but also 10% less cachesize (~ 10 MB).
Further work is needed to rip the whole benefits from this through, so
this is just the start.
Git-Dch: Ignore
|
|
Further abstracting our new ShowList allows to use it for containers of
strings as well giving us the option to implement an or-groups display
for the recommends and suggests lists which is a nice trick given that
it also helps with migrating the last remaining other cases of old
ShowList.
|
|
Housekeeping. This used to be embedded in apt-get directly, then moved
to into our (then new) private lib and now header and code get a proper
separation.
Git-Dch: Ignore
|
|
'files' is a bit too generic as a name for a command usually only used
programmatically (if at all) by developers, so instead of "wasting" this
generic name for this we use "indextargets" which is actually the name
of the datastructure the displayed data is stored in.
Along with this rename the config options are renamed accordingly.
|
|
C++11 adds the 'override' specifier to mark that a method is overriding
a base class method and error out if not. We hide it in the APT_OVERRIDE
macro to ensure that we keep compiling in pre-c++11 standards.
Reported-By: clang-modernize -add-override -override-macros
Git-Dch: Ignore
|
|
If all keyrings are simple keyrings we can merge the keyrings with cat
rather than doing a detour over gpg --export | --import (see #790665),
which means 'apt-key verify' can do without gpg and just use gpgv as
before the merging change.
We declare this gpgv usage explicit now in the dependencies. This isn't
a new dependency as gnupg as well as debian-archive-keyring depend on
and we used it before unconditionally, just that we didn't declare it.
The handling of the merged keyring needs to be slightly different as our
merged keyring can end up containing the same key multiple times, but at
least currently gpg does remove only the first occurrence with
--delete-keys, so we move the handling to a if one is gone, all are gone
rather than an (implicit) quid pro quo or even no effect.
Thanks: Daniel Kahn Gillmor for the suggestion
|
|
The output of gpg slightly changes in 2.1 which breaks the testcase, but
the real problem is that this branch introduces a new default keyring
format (which is called keybox) and mixing it with simple keyrings (the
previous default format) has various problems like failing in the keybox
to keyring import (#790665) or [older] gpgv versions not being able to
deal with keyboxes (and newer versions as well currently:
https://bugs.gnupg.org/gnupg/issue2025).
We fix this by being a bit more careful in who creates keyrings (aka: we
do it or we take a simple keyring as base) to ensure we always have a
keyring instead of a keybox. This way we can ensure that any version
combination of gpv/gpgv2 and gnupg/gnupg2 without doing explicit version
checks and use the same code for all of them.
Closes: 781042
|
|
It is sometimes handy to know how apt-key exactly called gpg, so adding
a pair of options to be able to see this if wanted is added. Two are
needed as some commands output is redirected to /dev/null, while sfor
others stdout is piped into another gpg call so in both cases you
wouldn't see all and hence you can choose.
Git-Dch: Ignore
|
|
Limits which key(s) can be used to sign a repository. Not immensely useful
from a security perspective all by itself, but if the user has
additional measures in place to confine a repository (like pinning) an
attacker who gets the key for such a repository is limited to its
potential and can't use the key to sign its attacks for an other (maybe
less limited) repository… (yes, this is as weak as it sounds, but having
the capability might come in handy for implementing other stuff later).
|
|
indexRecords was used to parse the Release file – mostly the hashes –
while metaIndex deals with downloading the Release file, storing all
indexes coming from this release and … parsing the Release file, but
this time mostly for the other fields.
That wasn't a problem in metaIndex as this was done in the type specific
subclass, but indexRecords while allowing to override the parsing method
did expect by default a specific format.
APT isn't really supporting different types at the moment, but this is
a violation of the abstraction we have everywhere else and, which is the
actual reason for this merge: Options e.g. coming from the sources.list
come to metaIndex naturally, which needs to wrap them up and bring them
into indexRecords, so the acquire system is told about it as they don't
get to see the metaIndex, but they don't really belong in indexRecords
as this is just for storing data loaded from the Release file… the
result is a complete mess.
I am not saying it is a lot prettier after the merge, but at least
adding new options is now slightly easier and there is just one place
responsible for parsing the Release file. That can't hurt.
|
|
We support arch= for a while, now we finally add lang= as well and as a
first simple way of controlling which targets to acquire also target=.
This asked for a redesign of the internal API of parsing and storing
information about 'deb' and 'deb-src' lines. As this API isn't visible
to the outside no damage done through.
Beside being a nice cleanup (= it actually does more in less lines) it
also provides us with a predictable order of architectures as provides
in the configuration rather than based on string sorting-order, so that
now the native architecture is parsed/displayed first. Observeable e.g.
in apt-get output.
|
|
Various small leaks here and there. Nothing particularily big, but still
good to fix. Found by the sanitizers while running our testcases.
Reported-By: gcc -fsanitize
Git-Dch: Ignore
|
|
C++11 slightly changes the API again to const_iterator, but we are find
with iterators in the C++03 style for now as long as they look and
behave equally to the methods of the standard containers.
Git-Dch: Ignore
|
|
Some of them modify the ABI, but given that we prepare a big one
already, these few hardly count for much.
Git-Dch: Ignore
|
|
|
|
Provided is a specialized acquire item which given a version can figure
out the correct URI to try by itself and if not provides an error
message alongside with static methods to get just the URI it would try
to download if it should just be displayed or similar such.
The URI is constructed as follows:
Release files can provide an URI template in the "Changelogs" field,
otherwise we lookup a configuration item based on the "Label" or
"Origin" of the Release file to get a (hopefully known) default value
for now. This template should contain the string CHANGEPATH which is
replaced with the information about the version we want the changelog
for (e.g. main/a/apt/apt_1.1). This middleway was choosen as this path
part was consistent over the three known implementations (+1 defunct),
while the rest of the URI varies widely between them.
The benefit of this construct is that it is now easy to get changelogs
for Debian packages on Ubuntu and vice versa – even at the moment where
the Changelogs field is present nowhere. Strictly better than what
apt-get had before as it would even fail to get changelogs from
security… Now it will notice that security identifies as Origin: Debian
and pick this setting (assuming again that no Changelogs field exists).
If on the other hand security would ship its changelogs in a different
location we could set it via the Label option overruling Origin.
Closes: 687147, 739854, 784027, 787190
|
|
Translation-* files are internally handled as PackageFiles which isn't
super nice, but giving them their own struct is a bit overkill so let it
be for the moment. They always appeared in the policy output because of
this through and now that they are properly linked to a ReleaseFile they
even display all the pinning information on them, but they don't contain
any packages which could be pinned… No problem, but useless and
potentially confusing output.
Adding a 'NoPackages' flag which can be set on those files and be used
in applications seems like a simple way to fix this display issue.
|
|
Selecting targets based on the Release they belong to isn't to
unrealistic. In fact, it is assumed to be the most used case so it is
made the default especially as this allows to bundle another thing we
have to be careful with: Filenames and only showing targets we have
acquired.
Closes: 752702
|
