summaryrefslogtreecommitdiff
path: root/apt-pkg/deb
AgeCommit message (Collapse)Author
2015-08-10no value for MultiArch field is 'no', not 'none'David Kalnischkies
Git-Dch: Ignore
2015-08-10drop obsolete explicit :none handling in pkgCacheGenDavid Kalnischkies
We archieve the same without the special handling now, so drop this code. Makes supporting this abdomination a little longer bearable as well. Git-Dch: Ignore
2015-08-10parse packages from all architectures into the cacheDavid Kalnischkies
Now that we can dynamically create dependencies and provides as needed rather than requiring to know with which architectures we will deal before running we can allow the listparser to parse all records rather than skipping records of "unknown" architectures. This can e.g. happen if a user has foreign architecture packages in his status file without dpkg knowing about this architecture (or apt configured in this way). A sideeffect is that now arch:all packages are (correctly) recorded as available from any Packages file, not just from the native one – which has its downsides for the resolver as mixed-arch source packages can appear in different architectures at different times, but that is the problem of the resolver and dealing with it in the parser is at best a hack (and also depends on a helpful repository). Another sideeffect is that his allows :none packages to appear in Packages files again as we don't do any kind of checks now, but given that they aren't really supported (anymore) by anyone we can live with that.
2015-08-10elimate duplicated code in pkgIndexFile subclassesDavid Kalnischkies
Trade deduplication of code for a bunch of new virtuals, so it is actually visible how the different indexes behave cleaning up the interface at large in the process. Git-Dch: Ignore
2015-08-10add volatile sources support in libapt-pkgDavid Kalnischkies
Sources are usually defined in sources.list (and co) and are pretty stable, but once in a while a frontend might want to add an additional "source" like a local .deb file to install this package (No support for 'real' sources being added this way as this is a multistep process). We had a hack in place to allow apt-get and apt to pull this of for a short while now, but other frontends are either left in the cold by this and/or the code for it looks dirty with FIXMEs plastering it and has on top of this also some problems (like including these 'volatile' sources in the srcpkgcache.bin file). So the biggest part in this commit is actually the rewrite of the cache generation as it is now potentially a three step process. The biggest problem with adding support now through is that this makes a bunch of previously mostly unusable by externs and therefore hidden classes public, so a bit of further tuneing on this now public API is in order…
2015-08-10just-in-time creation for (explicit) negative depsDavid Kalnischkies
Now that we deal with provides in a more dynamic fashion the last remaining problem is explicit dependencies like 'Conflicts: foo' which have to apply to all architectures, but creating them all at the same time requires us to know all architectures ending up in the cache which isn't needed to be the same set as all foreign architectures. The effect is visible already now through as this prevents the creation of a bunch of virtual packages for arch:all packages and as such also many dependencies, just not very visible if you don't look at the stats… Git-Dch Ignore
2015-08-10just-in-time creation for (implicit) ProvidesDavid Kalnischkies
Expecting the worst is easy to code, but has its disadvantages e.g. by creating package structures which otherwise would have never existed. By creating the provides instead at the time a package structure is added we are well prepared for the introduction of partial architectures, massive amounts of M-A:foreign (and :allowed) and co as far as provides are concerned at least. We have something relatively similar for dependencies already. Many tests are added for both M-A states and the code cleaned to properly support implicit provides for foreign architectures and architectures we 'just' happen to parse. Git-Dch: Ignore
2015-08-10hide implicit deps in apt-cache again by defaultDavid Kalnischkies
Before MultiArch implicits weren't a thing, so they were hidden by default by definition. Adding them for MultiArch solved many problems, but having no reliable way of detecting which dependency (and provides) is implicit or not causes problems everytime we want to output dependencies without confusing our observers with unneeded implementation details. The really notworthy point here is actually that we keep now a better record of how a dependency came to be so that we can later reason about it more easily, but that is hidden so deep down in the library internals that change is more the problems it solves than the change itself.
2015-08-10remove the compatibility markers for 4.13 abiDavid Kalnischkies
We aren't and we will not be really compatible again with the previous stable abi, so lets drop these markers (which never made it into a released version) for good as they have outlived their intend already. Git-Dch: Ignore
2015-08-10bunch of micro-optimizations for depcacheDavid Kalnischkies
DepCache functions are called a lot, so if we can squeeze some drops out of them for free we should do so. Takes also the opportunity to remove some whitespace errors from these functions. Git-Dch: Ignore
2015-08-10rename 'apt-get files' to 'apt-get indextargets'David Kalnischkies
'files' is a bit too generic as a name for a command usually only used programmatically (if at all) by developers, so instead of "wasting" this generic name for this we use "indextargets" which is actually the name of the datastructure the displayed data is stored in. Along with this rename the config options are renamed accordingly.
2015-08-10add c++11 override marker to overridden methodsDavid Kalnischkies
C++11 adds the 'override' specifier to mark that a method is overriding a base class method and error out if not. We hide it in the APT_OVERRIDE macro to ensure that we keep compiling in pre-c++11 standards. Reported-By: clang-modernize -add-override -override-macros Git-Dch: Ignore
2015-08-10allow individual targets to be kept compressedDavid Kalnischkies
There is an option to keep all targets (Packages, Sources, …) compressed for a while now, but the all-or-nothing approach is a bit limited for our purposes with additional targets as some of them are very big (Contents) and rarely used in comparison, so keeping them compressed by default can make sense, while others are still unpacked. Most interesting is the copy-change maybe: Copy is used by the acquire system as an uncompressor and it is hence expected that it returns the hashes for the "output", not the input. Now, in the case of keeping a file compressed, the output is never written to disk, but generated in memory and we should still validated it, so for compressed files copy is expected to return the hashes of the uncompressed file. We used to use the config option to enable on-the-fly decompress in the method, but in reality copy is never used in a way where it shouldn't decompress a compressed file to get its hashes, so we can save us the trouble of sending this information to the method and just do it always.
2015-08-10implement Signed-By option for sources.listDavid Kalnischkies
Limits which key(s) can be used to sign a repository. Not immensely useful from a security perspective all by itself, but if the user has additional measures in place to confine a repository (like pinning) an attacker who gets the key for such a repository is limited to its potential and can't use the key to sign its attacks for an other (maybe less limited) repository… (yes, this is as weak as it sounds, but having the capability might come in handy for implementing other stuff later).
2015-08-10add sources.list Check-Valid-Until and Valid-Until-{Max,Min} optionsDavid Kalnischkies
These options could be set via configuration before, but the connection to the actual sources is so strong that they should really be set in the sources.list instead – especially as this can be done a lot more specific rather than e.g. disabling Valid-Until for all sources at once. Valid-Until-* names are chosen instead of the Min/Max-ValidTime as this seems like a better name and their use in the wild is probably low enough that this isn't going to confuse anyone if we have to names for the same thing in different areas. In the longrun, the config options should be removed, but for now documentation hinting at the new options is good enough as these are the kind of options you set once across many systems with different apt versions, so the new way should work everywhere first before we deprecate the old way.
2015-08-10merge indexRecords into metaIndexDavid Kalnischkies
indexRecords was used to parse the Release file – mostly the hashes – while metaIndex deals with downloading the Release file, storing all indexes coming from this release and … parsing the Release file, but this time mostly for the other fields. That wasn't a problem in metaIndex as this was done in the type specific subclass, but indexRecords while allowing to override the parsing method did expect by default a specific format. APT isn't really supporting different types at the moment, but this is a violation of the abstraction we have everywhere else and, which is the actual reason for this merge: Options e.g. coming from the sources.list come to metaIndex naturally, which needs to wrap them up and bring them into indexRecords, so the acquire system is told about it as they don't get to see the metaIndex, but they don't really belong in indexRecords as this is just for storing data loaded from the Release file… the result is a complete mess. I am not saying it is a lot prettier after the merge, but at least adding new options is now slightly easier and there is just one place responsible for parsing the Release file. That can't hurt.
2015-08-10detect and error out on conflicting Trusted settingsDavid Kalnischkies
A specific trust state can be enforced via a sources.list option, but it effects all entries handled by the same Release file, not just the entry it was given on so we enforce acknowledgement of this by requiring the same value to be (not) set on all such entries.
2015-08-10support lang= and target= sources.list optionsDavid Kalnischkies
We support arch= for a while, now we finally add lang= as well and as a first simple way of controlling which targets to acquire also target=. This asked for a redesign of the internal API of parsing and storing information about 'deb' and 'deb-src' lines. As this API isn't visible to the outside no damage done through. Beside being a nice cleanup (= it actually does more in less lines) it also provides us with a predictable order of architectures as provides in the configuration rather than based on string sorting-order, so that now the native architecture is parsed/displayed first. Observeable e.g. in apt-get output.
2015-08-10fix memory leaks reported by -fsanitizeDavid Kalnischkies
Various small leaks here and there. Nothing particularily big, but still good to fix. Found by the sanitizers while running our testcases. Reported-By: gcc -fsanitize Git-Dch: Ignore
2015-08-10make all d-pointer * const pointersDavid Kalnischkies
Doing this disables the implicit copy assignment operator (among others) which would cause hovac if used on the classes as it would just copy the pointer, not the data the d-pointer points to. For most of the classes we don't need a copy assignment operator anyway and in many classes it was broken before as many contain a pointer of some sort. Only for our Cacheset Container interfaces we define an explicit copy assignment operator which could later be implemented to copy the data from one d-pointer to the other if we need it. Git-Dch: Ignore
2015-08-10apply various style suggestions by cppcheckDavid Kalnischkies
Some of them modify the ABI, but given that we prepare a big one already, these few hardly count for much. Git-Dch: Ignore
2015-06-16add d-pointer, virtual destructors and de-inline de/constructorsDavid Kalnischkies
To have a chance to keep the ABI for a while we need all three to team up. One of them missing and we might loose, so ensuring that they are available is a very tedious but needed task once in a while. Git-Dch: Ignore
2015-06-15hide Translation-* in 'apt-cache policy' outputDavid Kalnischkies
Translation-* files are internally handled as PackageFiles which isn't super nice, but giving them their own struct is a bit overkill so let it be for the moment. They always appeared in the policy output because of this through and now that they are properly linked to a ReleaseFile they even display all the pinning information on them, but they don't contain any packages which could be pinned… No problem, but useless and potentially confusing output. Adding a 'NoPackages' flag which can be set on those files and be used in applications seems like a simple way to fix this display issue.
2015-06-15populate the Architecture field for PackageFilesDavid Kalnischkies
This is mainly visible in the policy, so that you can now pin by b= and let it only effect Packages files of this architecture and hence the packages coming from it (which do not need to be from this architecture, but very likely are in a normal repository setup). If you should pin by architecture in this way is a different question… Closes: 687255
2015-06-15implement default apt-get file --release-info modeDavid Kalnischkies
Selecting targets based on the Release they belong to isn't to unrealistic. In fact, it is assumed to be the most used case so it is made the default especially as this allows to bundle another thing we have to be careful with: Filenames and only showing targets we have acquired. Closes: 752702
2015-06-12store Release files data in the CacheDavid Kalnischkies
We used to read the Release file for each Packages file and store the data in the PackageFile struct even through potentially many Packages (and Translation-*) files could use the same data. The point of the exercise isn't the duplicated data through. Having the Release files as first-class citizens in the Cache allows us to properly track their state as well as allows us to use the information also for files which aren't in the cache, but where we know to which Release file they belong (Sources are an example for this). This modifies the pkgCache structs, especially the PackagesFile struct which depending on how libapt users access the data in these structs can mean huge breakage or no visible change. As a single data point: aptitude seems to be fine with this. Even if there is breakage it is trivial to fix in a backportable way while avoiding breakage for everyone would be a huge pain for us. Note that not all PackageFile structs have a corresponding ReleaseFile. In particular the dpkg/status file as well as *.deb files have not. As these have only a Archive property need, the Component property takes over this duty and the ReleaseFile remains zero. This is also the reason why it isn't needed nor particularily recommended to change from PackagesFile to ReleaseFile blindly. Sticking with the earlier is usually the better option.
2015-06-11implement 'apt-get files' to access index targetsDavid Kalnischkies
Downloading additional files is only half the job. We still need a way to allow external tools to know where the files are they requested for download given that we don't want them to choose their own location. 'apt-get files' is our answer to this showing by default in a deb822 format information about each IndexTarget with the potential to filter the records based on lines and an option to change the output format. The command serves also as an example on how to get to this information via libapt.
2015-06-11use an enum instead of strings as IndexTarget::Option interfaceDavid Kalnischkies
Strings are easy to typo and we can keep the extensibility we require here with a simple enum we can append to without endangering ABI. Git-Dch: Ignore
2015-06-11use IndexTarget to get to IndexFileDavid Kalnischkies
Removes a bunch of duplicated code in the deb-specific parts. Especially the Description part is now handled centrally by IndexTarget instead of being duplicated to the derivations of IndexFile. Git-Dch: Ignore
2015-06-11show URI.Path in all acquire item descriptionsDavid Kalnischkies
It is a rather strange sight that index items use SiteOnly which strips the Path, while e.g. deb files are downloaded with NoUserPassword which does not. Important to note here is that for the file transport Path is pretty important as there is no Host which would be displayed by Site, which always resulted in "interesting" unspecific errors for "file:". Adding a 'middle' ground between the two which does show the Path but potentially modifies it (it strips a pending / at the end if existing) solves this "file:" issue, syncs the output and in the end helps to identify which file is meant exactly in progress output and co as a single site can have multiple repositories in different paths.
2015-06-10rename Calculate- to GetIndexTargets and use it as official APIDavid Kalnischkies
We need a general way to get from a sources.list entry to IndexTargets and with this change we can move from pkgSourceList over the list of metaIndexes it includes to the IndexTargets each metaIndex can have. Git-Dch: Ignore
2015-06-10stop using IndexTarget pointers which are never freedDavid Kalnischkies
Creating and passing around a bunch of pointers of IndexTargets (and of a vector of pointers of IndexTargets) is probably done to avoid the 'costly' copy of container, but we are really not in a timecritical operation here and move semantics will help us even further in the future. On the other hand we never do a proper cleanup of these pointers, which is very dirty, even if structures aren't that big… The changes will effecting many items only effect our own hidden class, so we can do that without fearing breaking interfaces or anything. Git-Dch: Ignore
2015-06-10store all targets data in IndexTarget structDavid Kalnischkies
We still need an API for the targets, so slowly prepare the IndexTargets to let them take this job. Git-Dch: Ignore
2015-06-10abstract the code to iterate over all targets a bitDavid Kalnischkies
We have two places in the code which need to iterate over targets and do certain things with it. The first one is actually creating these targets for download and the second instance pepares certain targets for reading. Git-Dch: Ignore
2015-06-09configureable acquire targets to download additional filesDavid Kalnischkies
First pass at making the acquire system capable of downloading files based on configuration rather than hardcoded entries. It is now possible to instruct 'deb' and 'deb-src' sources.list lines to download more than just Packages/Translation-* and Sources files. Details on how to do that can be found in the included documentation file.
2015-06-09rework hashsum verification in the acquire systemDavid Kalnischkies
Having every item having its own code to verify the file(s) it handles is an errorprune process and easy to break, especially if items move through various stages (download, uncompress, patching, …). With a giant rework we centralize (most of) the verification to have a better enforcement rate and (hopefully) less chance for bugs, but it breaks the ABI bigtime in exchange – and as we break it anyway, it is broken even harder. It shouldn't effect most frontends as they don't deal with the acquire system at all or implement their own items, but some do and will need to be patched (might be an opportunity to use apt on-board material). The theory is simple: Items implement methods to decide if hashes need to be checked (in this stage) and to return the expected hashes for this item (in this stage). The verification itself is done in worker message passing which has the benefit that a hashsum error is now a proper error for the acquire system rather than a Done() which is later revised to a Failed().
2015-05-11stop depending on copy-on-write for std::stringDavid Kalnischkies
In 66c3875df391b1120b43831efcbe88a78569fbfe we workaround/fixed a problem where the code makes the assumption that the compiler uses copy-on-write implementations for std::string. Turns out that for c++11 compatibility gcc >= 5 will stop doing this by default.
2015-04-19Merge branch 'debian/jessie' into debian/experimentalDavid Kalnischkies
Conflicts: apt-pkg/acquire-item.cc cmdline/apt-key.in methods/https.cc test/integration/test-apt-key test/integration/test-multiarch-foreign
2015-04-12parse specific-arch dependencies correctly on single-arch systemsDavid Kalnischkies
On single-arch the parsing was creating groupnames like 'apt:amd64' even through it should be 'apt' and a package in it belonging to architecture amd64. The result for foreign architectures was as expected: The dependency isn't satisfiable, but for native architecture it means the wrong package (ala apt:amd64:amd64) is linked so this is also not satisfiable, which is very much not expected. No longer excluding single-arch from this codepath allows the generation of the correct links, which still link to non-exisiting packages for foreign dependencies, but natives link to the expected native package just as if no architecture was given. For negative arch-specific dependencies ala Conflicts this matter was worse as apt will believe there isn't a Conflict to resolve, tricking it into calculating a solution dpkg will refuse. Architecture specific positive dependencies are rare in jessie – the only one in amd64 main is foreign –, negative dependencies do not even exist. Neither class has a native specimen, so no package in jessie is effected by this bug, but it might be interesting for stretch upgrades. This also means the regression potential is very low. Closes: 777760
2015-04-07fix crash in order writing in pkgDPkgPM::WriteApportReport()Michael Vogt
libapt can be configured to write various bits of information to a file creating a report via apport. This is disabled by default in Debian and apport residing only in /experimental so far, but Ubuntu and other derivatives have this (in some versions) enabled by default and there is no regression potentially here. The crash is caused by a mismatch of operations vs. strings for operations, so adding the missing strings for these operations solves the problem. [commit message by David Kalnischkies] LP: #1436626
2015-03-16parse arch-qualified Provides correctlyHelmut Grohne
The underlying problem is that libapt-pkg does not correctly parse these provides. Internally, it creates a version named "baz:i386" with architecture amd64. Of course, such a package name is invalid and thus this version is completely inaccessible. Thus, this bug should not cause apt to accept a broken situation as valid. Nevertheless, it prevents using architecture qualified depends. Closes: 777071
2015-03-16fix some new compiler warnings reported by gcc-5David Kalnischkies
Git-Dch: Ignore
2015-03-16properly implement pkgRecord::Parser for *.deb filesDavid Kalnischkies
Implementing FileName() works for most cases for us, but other frontends might need more and even for us its not very stable as the normal Jump() implementation is pretty bad on a deb file and produce errors on its own at times. So, replacing this makeshift with a complete implementation by mostly just shuffling code around.
2015-03-16merge debian/sid into debian/experimentalDavid Kalnischkies
2014-12-23pass-through stdin fd instead of content if not a terminalDavid Kalnischkies
Commit 299aea924ccef428219ed6f1a026c122678429e6 fixes the problem of not logging terminal in case stdin & stdout are not a terminal. The problem is that we are then trying to pass-through stdin content by reading from the apt-process stdin and writing it to the stdin of the child (dpkg), which works great for users who can control themselves, but pipes and co are a bit less forgiving causing us to pass everything to the first child process, which if the sending part of the pipe is e.g. 'yes' we will never see the end of it (as the pipe is full at some point and further writing blocks). There is a simple solution for that of course: If stdin isn't a terminal, we us the apt-process stdin as stdin for the child directly (We don't do this if it is a terminal to be able to save the typed input in the log). Closes: 773061
2014-12-23always run 'dpkg --configure -a' at the end of our dpkg callingsDavid Kalnischkies
dpkg checks now for dependencies before running triggers, so that packages can now end up in trigger states (especially those we are not touching at all with our calls) after apt is done running. The solution to this is trivial: Just tell dpkg to configure everything after we have (supposely) configured everything already. In the worst case this means dpkg will have to run a bunch of triggers, usually it will just do nothing though. The code to make this happen was already available, so we just flip a config option here to cause it to be run. This way we can keep pretending that triggers are an implementation detail of dpkg. --triggers-only would supposely work as well, but --configure is more robust in regards to future changes to dpkg and something we will hopefully make use of in future versions anyway (as it was planed at the time this and related options were implemented). Note that dpkg currently has a workaround implemented to allow upgrades to jessie to be clean, so that the test works before and after. Also note that test (compared to the one in the bug) drops the await test as its is considered a loop by dpkg now. Closes: 769609
2014-12-23do not make PTY slave the controlling terminalDavid Kalnischkies
If we have no controlling terminal opening a terminal will make this terminal our controller, which is a serious problem if this happens to be the pseudo terminal we created to run dpkg in as we will close this terminal at the end hanging ourself up in the process… The offending open is the one we do to have at least one slave fd open all the time, but for good measure, we apply the flag also to the slave fd opening in the child process as we set the controlling terminal explicitely here. This is a regression from 150bdc9ca5d656f9fba94d37c5f4f183b02bd746 with the slight twist that this usecase was silently broken before in that it wasn't logging the output in term.log (as a pseudo terminal wasn't created). Closes: 772641
2014-12-10do not make PTY slave the controlling terminalDavid Kalnischkies
If we have no controlling terminal opening a terminal will make this terminal our controller, which is a serious problem if this happens to be the pseudo terminal we created to run dpkg in as we will close this terminal at the end hanging ourself up in the process… The offending open is the one we do to have at least one slave fd open all the time, but for good measure, we apply the flag also to the slave fd opening in the child process as we set the controlling terminal explicitely here. This is a regression from 150bdc9ca5d656f9fba94d37c5f4f183b02bd746 with the slight twist that this usecase was silently broken before in that it wasn't logging the output in term.log (as a pseudo terminal wasn't created). Closes: 772641
2014-12-07always run 'dpkg --configure -a' at the end of our dpkg callingsDavid Kalnischkies
dpkg checks now for dependencies before running triggers, so that packages can now end up in trigger states (especially those we are not touching at all with our calls) after apt is done running. The solution to this is trivial: Just tell dpkg to configure everything after we have (supposely) configured everything already. In the worst case this means dpkg will have to run a bunch of triggers, usually it will just do nothing though. The code to make this happen was already available, so we just flip a config option here to cause it to be run. This way we can keep pretending that triggers are an implementation detail of dpkg. --triggers-only would supposely work as well, but --configure is more robust in regards to future changes to dpkg and something we will hopefully make use of in future versions anyway (as it was planed at the time this and related options were implemented). Closes: 769609
2014-12-07correct architecture detection for 'rc' packages for purgeDavid Kalnischkies
We were already considering these cases, but the code was flawed, so that packages changing architectures are incorrectly handled and hence the wrong architecture is used to call dpkg with, so that dpkg says the package isn't installed (which it isn't for the requested architecture). Closes: 770898