summaryrefslogtreecommitdiff
path: root/cmdline
AgeCommit message (Collapse)Author
2015-09-14implement dpkgs vision of interpreting pkg:<arch> dependenciesDavid Kalnischkies
How the Multi-Arch field and pkg:<arch> dependencies interact was discussed at DebConf15 in the "MultiArch BoF". dpkg and apt (among other tools like dose) had a different interpretation in certain scenarios which we resolved by agreeing on dpkg view – and this commit realizes this agreement in code. As was the case so far libapt sticks to the idea of trying to hide MultiArch as much as possible from individual frontends and instead translates it to good old SingleArch. There are certainly situations which can be improved in frontends if they know that MultiArch is upon them, but these are improvements – not necessary changes needed to unbreak a frontend. The implementation idea is simple: If we parse a dependency on foo:amd64 the dependency is formed on a package 'foo:amd64' of arch 'any'. This package is provided by package 'foo' of arch 'amd64', but not by 'foo' of arch 'i386'. Both of those foo packages provide each other through (assuming foo is M-A:foreign) to allow a dependency on 'foo' to be satisfied by either foo of amd64 or i386. Packages can also declare to provide 'foo:amd64' which is translated to providing 'foo:amd64:any' as well. This indirection over provides was chosen as the alternative would be to teach dependency resolvers how to deal with architecture specific dependencies – which violates the design idea of avoiding resolver changes, especially as architecture-specific dependencies are a cornercase with quite a few subtil rules. Handling it all over versioned provides as we already did for M-A in general seems much simpler as it just works for them. This switch to :any has actually a "surprising" benefit as well: Even frontends showing a package name via .Name() [which doesn't show the architecture] will display the "architecture" for dependencies in which it was explicitely requested, while we will not show the 'strange' :any arch in FullName(true) [= pretty-print] either. Before you had to specialcase these and by default you wouldn't get these details shown. The only identifiable disadvantage is that this complicates error reporting and handling. apt-get's ShowBroken has existing problems with virtual packages [it just shows the name without any reason], so that has to be worked on eventually. The other case is that detecting if a package is completely unknown or if it was at least referenced somewhere needs to acount for this "split" – not that it makes a practical difference which error is shown… but its one of the improvements possible.
2015-09-01use clock() as source for SRV randomnessDavid Kalnischkies
Initializing a random number generator with the time since epoch could be good enough, but reaches its limits in test code as the 100 iterations might very well happen in the same second and hence the seed number is always the same… clock() has a way lower resolution so it changes more often and not unimportant: If many users start the update at the same time it isn't to unlikely the SRV record will be ordered in the same second choosing the same for them all, but it seems less likely that the exact same clock() time has passed for them. And if I have to touch this, lets change a few other things as well to make me and/or compilers a bit happier (clang complained about the usage of a GNU extension in the testcase for example).
2015-08-31if file is inaccessible for _apt, disable privilege drop in acquireDavid Kalnischkies
We had a very similar method previously for our own private usage, but with some generalisation we can move this check into the acquire system proper so that all frontends profit from this compatibility change. As we are disabling a security feature here a warning is issued and frontends are advised to consider reworking their download logic if possible. Note that this is implemented as an all or nothing situation: We can't just (not) drop privileges for a subset of the files in a fetcher, so in case you have to download some files with and some without you need to use two fetchers.
2015-08-29implement indextargets option 'DefaultEnabled'David Kalnischkies
Some targets like Contents-udeb are special-needs targets. Shipping the configuration snippet for them is okay, but they shouldn't be downloaded by default. Forcing the user to enable targets by uncommenting targets is wrong and this would still not really solve the problem completely as even if you want to download some -udebs it will probably not be for all sources you have enabled, so having the possibility of disabling a target by default, but giving the user the option to enable it on a per-source entry basis is better.
2015-08-28implement PDiff patching for compressed filesDavid Kalnischkies
Some additional files like 'Contents' are very big and should therefore kept compressed on the disk, which apt-file did in the past. It also implemented pdiff patching of these files by un- and recompressing these files on-the-fly, with this commit we can do the same – but we can do this in both pdiff patching styles (client and server merging) and secured by hashes. Hashes are in so far slightly complicated as we can't compare the hashes of the compressed files as we might compress them differently than the server would (different compressor versions, options, …), so we must compare the hashes of the uncompressed content. While this commit has changes in public headers, the classes it changes are marked as hidden, so nobody can use them directly, which means the ABI break is internal only.
2015-08-27sources.list and indextargets option for pdiffsDavid Kalnischkies
Disabling pdiffs can be useful occasionally, like if you have a fast local mirror where the download doesn't matter, but still want to use it for non-local mirrors. Also, some users might prefer it to only use it for very big indextargets like Contents.
2015-08-27use always priv-dropping for changelog download as rootDavid Kalnischkies
First of, the temporary directory we download the changelog to needs to be owned by _apt, but that also means that we don't need to check if we could/should drop privs as the download happens to a dedicated tempdir and only after that it is moved to its final location by a privileged user.
2015-08-27fix various typos reported by codespellDavid Kalnischkies
Reported-By: codespell
2015-08-20Merge remote-tracking branch 'mvo/feature/srv-records' into debian/experimentalMichael Vogt
2015-08-18Add support for "apt-cache showsrc --only-source srcpkgname"Michael Vogt
Thanks: Steve Slangasek for the suggestion Closes: 695633
2015-08-18Provide "apt-get full-upgrade" to match "apt full-upgrade"Michael Vogt
2015-08-18cleanupMichael Vogt
2015-08-18Merge remote-tracking branch 'upstream/debian/experimental' into ↵Michael Vogt
feature/srv-records
2015-08-18Merge branch 'debian/experimental' into feature/srv-recordsMichael Vogt
Conflicts: cmdline/apt-helper.cc cmdline/makefile
2015-08-16Show full package records in apt-cache search -fKusanagi Kouichi
This just changes the DoSearch code to use DisplayRecord to display the record when the full record is requested. Closes: #660851 [jak@debian.org: Wrote the commit message]
2015-08-16When looking if Provides match, OR them with the normal patchesJulian Andres Klode
Simply overriding the value caused patterns that previously matched a real package name to not match anymore. Closes: #760868
2015-08-14apt-get: allow non-root --print-uris build-depJulian Andres Klode
Closes: #283400
2015-08-14apt: Add autoremove and auto-remove commandsJulian Andres Klode
2015-08-14Make auto-remove and auto-clean aliases for the versions without -Julian Andres Klode
Some people type them instead of autoremove and autoclean, so make them happy. Closes: #274159 Makes-Happy: Ansgar
2015-08-14apt-cache: Show an error if stats gets any argumentsJulian Andres Klode
Closes: #153161
2015-08-14Merge branch 'debian/experimental' of https://github.com/DonKult/apt into ↵Julian Andres Klode
debian/experimental
2015-08-13Deprecate SPtrArray<T> and convert everyone to unique_ptr<T[]>Julian Andres Klode
More standardization
2015-08-13Mark SPtr as deprecated, and convert users to std::unique_ptrJulian Andres Klode
Switch to std::unique_ptr, as this is safer than SPtr.
2015-08-12apt-cache: Improve translateability of the "with priority" thingJulian Andres Klode
Gbp-Dch: ignore
2015-08-12apt-cache: Modify policy output to use per-version pinsJulian Andres Klode
Also optionally enable old output by setting APT::Policy=0.
2015-08-12Replace all "press enter" occurrences with "press [Enter]"Luca Bruno
Thanks: Andre Felipe Machado for initial patch Closes: 414848
2015-08-11apt-get: Do not include apt-pkg/indexrecords.hJulian Andres Klode
It's gone. Gbp-Dch: ignore
2015-08-10no value for MultiArch field is 'no', not 'none'David Kalnischkies
Git-Dch: Ignore
2015-08-10elimate duplicated code in pkgIndexFile subclassesDavid Kalnischkies
Trade deduplication of code for a bunch of new virtuals, so it is actually visible how the different indexes behave cleaning up the interface at large in the process. Git-Dch: Ignore
2015-08-10just-in-time creation for (implicit) ProvidesDavid Kalnischkies
Expecting the worst is easy to code, but has its disadvantages e.g. by creating package structures which otherwise would have never existed. By creating the provides instead at the time a package structure is added we are well prepared for the introduction of partial architectures, massive amounts of M-A:foreign (and :allowed) and co as far as provides are concerned at least. We have something relatively similar for dependencies already. Many tests are added for both M-A states and the code cleaned to properly support implicit provides for foreign architectures and architectures we 'just' happen to parse. Git-Dch: Ignore
2015-08-10hide implicit deps in apt-cache again by defaultDavid Kalnischkies
Before MultiArch implicits weren't a thing, so they were hidden by default by definition. Adding them for MultiArch solved many problems, but having no reliable way of detecting which dependency (and provides) is implicit or not causes problems everytime we want to output dependencies without confusing our observers with unneeded implementation details. The really notworthy point here is actually that we keep now a better record of how a dependency came to be so that we can later reason about it more easily, but that is hidden so deep down in the library internals that change is more the problems it solves than the change itself.
2015-08-10remove the compatibility markers for 4.13 abiDavid Kalnischkies
We aren't and we will not be really compatible again with the previous stable abi, so lets drop these markers (which never made it into a released version) for good as they have outlived their intend already. Git-Dch: Ignore
2015-08-10split-up Dependency structDavid Kalnischkies
Having dependency data separated from the link between version/package and the dependency allows use to work on sharing the depdency data a bit as it turns out that many dependencies are in fact duplicates. How many are duplicates various heavily with the sources configured, but for a single Debian release the ballpark is 2 duplicates for each dependency already (e.g. libc6 counts 18410 dependencies, but only 45 unique). Add more releases and the duplicates count only rises to get ~6 for 3 releases. For each architecture a user has configured which given the shear number of dependencies amounts to MBs of duplication. We can cut down on this number, but pay a heavy price for it: In my many releases(3) + architectures(3) test we have a 10% (~ 0.5 sec) increase in cache creationtime, but also 10% less cachesize (~ 10 MB). Further work is needed to rip the whole benefits from this through, so this is just the start. Git-Dch: Ignore
2015-08-10show or-groups in not-installed recommends and suggests listsDavid Kalnischkies
Further abstracting our new ShowList allows to use it for containers of strings as well giving us the option to implement an or-groups display for the recommends and suggests lists which is a nice trick given that it also helps with migrating the last remaining other cases of old ShowList.
2015-08-10headers are for declarations onlyDavid Kalnischkies
Housekeeping. This used to be embedded in apt-get directly, then moved to into our (then new) private lib and now header and code get a proper separation. Git-Dch: Ignore
2015-08-10rename 'apt-get files' to 'apt-get indextargets'David Kalnischkies
'files' is a bit too generic as a name for a command usually only used programmatically (if at all) by developers, so instead of "wasting" this generic name for this we use "indextargets" which is actually the name of the datastructure the displayed data is stored in. Along with this rename the config options are renamed accordingly.
2015-08-10add c++11 override marker to overridden methodsDavid Kalnischkies
C++11 adds the 'override' specifier to mark that a method is overriding a base class method and error out if not. We hide it in the APT_OVERRIDE macro to ensure that we keep compiling in pre-c++11 standards. Reported-By: clang-modernize -add-override -override-macros Git-Dch: Ignore
2015-08-10merge keyrings with cat instead of gpg in apt-keyDavid Kalnischkies
If all keyrings are simple keyrings we can merge the keyrings with cat rather than doing a detour over gpg --export | --import (see #790665), which means 'apt-key verify' can do without gpg and just use gpgv as before the merging change. We declare this gpgv usage explicit now in the dependencies. This isn't a new dependency as gnupg as well as debian-archive-keyring depend on and we used it before unconditionally, just that we didn't declare it. The handling of the merged keyring needs to be slightly different as our merged keyring can end up containing the same key multiple times, but at least currently gpg does remove only the first occurrence with --delete-keys, so we move the handling to a if one is gone, all are gone rather than an (implicit) quid pro quo or even no effect. Thanks: Daniel Kahn Gillmor for the suggestion
2015-08-10support gpg 2.1.x in apt-keyDavid Kalnischkies
The output of gpg slightly changes in 2.1 which breaks the testcase, but the real problem is that this branch introduces a new default keyring format (which is called keybox) and mixing it with simple keyrings (the previous default format) has various problems like failing in the keybox to keyring import (#790665) or [older] gpgv versions not being able to deal with keyboxes (and newer versions as well currently: https://bugs.gnupg.org/gnupg/issue2025). We fix this by being a bit more careful in who creates keyrings (aka: we do it or we take a simple keyring as base) to ensure we always have a keyring instead of a keybox. This way we can ensure that any version combination of gpv/gpgv2 and gnupg/gnupg2 without doing explicit version checks and use the same code for all of them. Closes: 781042
2015-08-10enhance apt-key debugging optionsDavid Kalnischkies
It is sometimes handy to know how apt-key exactly called gpg, so adding a pair of options to be able to see this if wanted is added. Two are needed as some commands output is redirected to /dev/null, while sfor others stdout is piped into another gpg call so in both cases you wouldn't see all and hence you can choose. Git-Dch: Ignore
2015-08-10implement Signed-By option for sources.listDavid Kalnischkies
Limits which key(s) can be used to sign a repository. Not immensely useful from a security perspective all by itself, but if the user has additional measures in place to confine a repository (like pinning) an attacker who gets the key for such a repository is limited to its potential and can't use the key to sign its attacks for an other (maybe less limited) repository… (yes, this is as weak as it sounds, but having the capability might come in handy for implementing other stuff later).
2015-08-10merge indexRecords into metaIndexDavid Kalnischkies
indexRecords was used to parse the Release file – mostly the hashes – while metaIndex deals with downloading the Release file, storing all indexes coming from this release and … parsing the Release file, but this time mostly for the other fields. That wasn't a problem in metaIndex as this was done in the type specific subclass, but indexRecords while allowing to override the parsing method did expect by default a specific format. APT isn't really supporting different types at the moment, but this is a violation of the abstraction we have everywhere else and, which is the actual reason for this merge: Options e.g. coming from the sources.list come to metaIndex naturally, which needs to wrap them up and bring them into indexRecords, so the acquire system is told about it as they don't get to see the metaIndex, but they don't really belong in indexRecords as this is just for storing data loaded from the Release file… the result is a complete mess. I am not saying it is a lot prettier after the merge, but at least adding new options is now slightly easier and there is just one place responsible for parsing the Release file. That can't hurt.
2015-08-10support lang= and target= sources.list optionsDavid Kalnischkies
We support arch= for a while, now we finally add lang= as well and as a first simple way of controlling which targets to acquire also target=. This asked for a redesign of the internal API of parsing and storing information about 'deb' and 'deb-src' lines. As this API isn't visible to the outside no damage done through. Beside being a nice cleanup (= it actually does more in less lines) it also provides us with a predictable order of architectures as provides in the configuration rather than based on string sorting-order, so that now the native architecture is parsed/displayed first. Observeable e.g. in apt-get output.
2015-08-10fix memory leaks reported by -fsanitizeDavid Kalnischkies
Various small leaks here and there. Nothing particularily big, but still good to fix. Found by the sanitizers while running our testcases. Reported-By: gcc -fsanitize Git-Dch: Ignore
2015-08-10cleanup Container.erase API to look more like std::containersDavid Kalnischkies
C++11 slightly changes the API again to const_iterator, but we are find with iterators in the C++03 style for now as long as they look and behave equally to the methods of the standard containers. Git-Dch: Ignore
2015-08-10apply various style suggestions by cppcheckDavid Kalnischkies
Some of them modify the ABI, but given that we prepare a big one already, these few hardly count for much. Git-Dch: Ignore
2015-08-10apt-cache: Change version pin output to use per-version pinsJulian Andres Klode
2015-06-15provide a public interface for acquiring changelogsDavid Kalnischkies
Provided is a specialized acquire item which given a version can figure out the correct URI to try by itself and if not provides an error message alongside with static methods to get just the URI it would try to download if it should just be displayed or similar such. The URI is constructed as follows: Release files can provide an URI template in the "Changelogs" field, otherwise we lookup a configuration item based on the "Label" or "Origin" of the Release file to get a (hopefully known) default value for now. This template should contain the string CHANGEPATH which is replaced with the information about the version we want the changelog for (e.g. main/a/apt/apt_1.1). This middleway was choosen as this path part was consistent over the three known implementations (+1 defunct), while the rest of the URI varies widely between them. The benefit of this construct is that it is now easy to get changelogs for Debian packages on Ubuntu and vice versa – even at the moment where the Changelogs field is present nowhere. Strictly better than what apt-get had before as it would even fail to get changelogs from security… Now it will notice that security identifies as Origin: Debian and pick this setting (assuming again that no Changelogs field exists). If on the other hand security would ship its changelogs in a different location we could set it via the Label option overruling Origin. Closes: 687147, 739854, 784027, 787190
2015-06-15hide Translation-* in 'apt-cache policy' outputDavid Kalnischkies
Translation-* files are internally handled as PackageFiles which isn't super nice, but giving them their own struct is a bit overkill so let it be for the moment. They always appeared in the policy output because of this through and now that they are properly linked to a ReleaseFile they even display all the pinning information on them, but they don't contain any packages which could be pinned… No problem, but useless and potentially confusing output. Adding a 'NoPackages' flag which can be set on those files and be used in applications seems like a simple way to fix this display issue.
2015-06-15implement default apt-get file --release-info modeDavid Kalnischkies
Selecting targets based on the Release they belong to isn't to unrealistic. In fact, it is assumed to be the most used case so it is made the default especially as this allows to bundle another thing we have to be careful with: Filenames and only showing targets we have acquired. Closes: 752702