summaryrefslogtreecommitdiff
path: root/apt-pkg/edsp.cc
AgeCommit message (Collapse)Author
2016-06-27eipp: enable xz-compressed scenario loggingDavid Kalnischkies
In 385d9f2f23057bc5808b5e013e77ba16d1c94da4 I implemented the storage of scenario files based on enabling this by default for EIPP, but I implemented it first optionally for EDSP to have it independent. The reasons mentioned in the earlier commit (debugging and bugreports) obviously apply here, especially as EIPP solutions aren't user approved, nearly impossible to verify before starting the execution and at the time of error the scenario has changed already, so that reproducing the issue becomes hard(er).
2016-06-27eipp: rename stanza 'Install' to 'Unpack'David Kalnischkies
Freeing 'Install' for future use as an interface for "dpkg --install", which is currently not used by any existent planer, so the implementation of it itself will be delayed until then.
2016-06-27eipp: add Allow-Temporary-Remove-of-EssentialsDavid Kalnischkies
A rather special need option, but the internal planer supports this and we have a testcase for it & sometimes it is hit (as a bug through). The option itself mostly serves as a reminder for implementors that they should be careful with removes and especially temporary removes if they perform any.
2016-06-27eipp: implement Immediate-Configuration flagDavid Kalnischkies
APT has 3 modes: no immediate configuration, all packages are configured immediately and its default mode of configuring essentials and pseudo-essentials immediately only. While this seems like a job of different planers at first, it might be handy to have it as an option, too, in case a planer (like apts internal one) supports different modes where the introduction of individual planers would be counter intuitive.
2016-06-27eipp: properly handle arch-specific providesDavid Kalnischkies
The generation of the EIPP request was a bit to strict not generation what would actually be needed to be part of the scenario.
2016-06-27eipp: make no difference between remove & purgeDavid Kalnischkies
For the order there is no inherent difference between delete or purge, so we don't tell the planer about this and instead decide in apt if a package should be purged or not which also allows us to not tell the planer about rc-only purges as we can trivially do this on our on as there is no need to plan such purges.
2016-06-27eipp: provide the internal planer as an external oneDavid Kalnischkies
Testing the current implementation can benefit from being able to be feed an EIPP request and produce a fully compliant response. It is also a great test for EIPP in general.
2016-06-27eipp: output at most two versions per packageDavid Kalnischkies
We can trim generation time and size of the EIPP scenario considerable if we we avoid telling the planers about "uninteresting" packages. This is one of the simpler but already very effective reductions: Do not tell planers about versions which are neither installed nor are to be installed as they have no effect on the plan we don't need to tell the planer about them. EDSP solvers need to know about all versions for better choice and error messages, but planers really don't. Git-Dch: Ignore
2016-06-27eipp: implement version 0.1 of the protocolDavid Kalnischkies
The very first step in introducing the "external installation planer protocol" (short: EIPP) as part of my GSoC2016 project. The description reads: APT-based tools like apt-get, aptitude, synaptic, … work with the user to figure out how their system should look like after they are done installing/removing packages and their dependencies. The actual installation/removal of packages is done by dpkg with the constrain that dependencies must be fulfilled at any point in time (e.g. to run maintainer scripts). Historically APT has a super micro-management approach to this task which hasn't aged that well over the years mostly ignoring changes in dpkg and growing into an unmaintainable mess hardly anyone can debug and everyone fears to touch – especially as more and more requirements are tacked onto it like handling cycles and triggers, dealing with "important" packages first, package sources on removable media, touch minimal groups to be able to interrupt the process if needed (e.g. unattended-upgrades) which not only sky-rocket complexity but also can be mutually exclusive as you e.g. can't have minimal groups and minimal trigger executions at the same time.
2016-06-10don't leak EDSP solver output fdDavid Kalnischkies
2016-06-08edsp: drop privileges before executing solversDavid Kalnischkies
Most (if not all) solvers should be able to run perfectly fine without root privileges as they get the entire state they are supposed to work on via stdin and do not perform any action directly, but just pass suggestions on via stdout. The new default is to run them all as _apt hence, but each solver can configure another user if it chooses/must. The security benefits are minimal at best, but it helps preventing silly mistakes (see 35f3ed061f10a25a3fb28bc988fddbb976344c4d) and that is always good. Note that our 'apt' and 'dump' solver already dropped privileges if they had them.
2016-06-08edsp: if logging is requested, do it for internal, tooDavid Kalnischkies
Create and log the EDSP(like) request even if we use the internal resolver.
2016-06-08edsp: optionally store a compressed copy of the last scenarioDavid Kalnischkies
For bugreports and co it could be handy to have the scenario and all the settings used in it around later for inspection for EDSP like protocols. EDSP might not be the most interesting as the user can still interrupt the process before the solution is applied and users tend to have an opinion on the "rightness" of a solution, so it is disabled by default.
2016-06-06get right of the hardcoded string-length comparesDavid Kalnischkies
With have better ways to compare, manipulate and work with strings, so use it instead of counting string length by hand with is a wonder it hasn't failed yet. Ignoreable from a changelog perspective as there is no behaviour change. Git-Dch: Ignore
2016-06-04edsp: use a stanza based interface for solution writingDavid Kalnischkies
EDSP had a WriteSolution method to write out the entire solution based on the inspection of a given pkgDepCache, but that is rather inflexible both for EDSP itself and for other EDSP like-protocols. It seems better to use a smaller scope in printing just a single stanza based on a given version as there is more reuse potential.
2016-06-04edsp: use an ID mapping for the internal solverDavid Kalnischkies
Currently an EDSP solver gets send basically all versions which means the absolute count is the same, but that might not be true forever (and with the skipping of rc-only versions it kinda is already) and even if it were true, segfaulting on bad input seems wrong.
2016-05-20refactor EDSP code into EDSP and EDSP-like partsDavid Kalnischkies
No real code change, just moving code around heavily to decouple the EDSP specific parts from those we can reuse for EDSP-like protocols. Git-Dch: Ignore
2016-05-20edsp: use a limited scenario based on bool-arrayDavid Kalnischkies
Its more space and runtime efficient to use a boolean set instead of a CacheSet-based implementation. Git-Dch: Ignore
2016-05-20edsp: add Forbid-{New-Install,Remove} and Upgrade-AllDavid Kalnischkies
This allows to differentiate properly between 'apt-get upgrade', 'apt upgrade' and 'apt full-upgrade'.
2016-05-20avoid triggering gcc's -Wunsafe-loop-optimizations in EDSPDavid Kalnischkies
apt/apt-pkg/edsp.cc: In function ‘bool EDSP::WriteLimitedScenario(pkgDepCache&, FILE*, const PackageSet&, OpProgress*)’: apt/apt-pkg/edsp.cc:245:56: warning: cannot optimize loop, the loop counter may overflow [-Wunsafe-loop-optimizations] std::string dependencies[pkgCache::Dep::Enhances + 1]; ^ Using a std::array to silence gcc as well as as a code improvement feels right here. Git-Dch: Ignore
2016-05-20edsp: try harder to not generate unneeded error messagesDavid Kalnischkies
The &= introduced in the EDSP-FileFd conversion isn't working to full satisfaction for multiple && clauses as the && has a higher binding than &= has, so that the methods were called even through they shouldn't have because of previous errors. Using variadic functions we can solve this in a slightly cleaner way bringing down the amount of 'broken pipe' errors for the error case of the dump resolver substantially. Git-Dch: Ignore
2016-05-20convert EDSP to be based on FileFd instead of FILE*David Kalnischkies
I doubt there is any non-src:apt usage of these interfaces.
2016-05-12edsp: warn if unexpected stanzas appear in the solutionDavid Kalnischkies
Unexpected are for examples removal requests for versions which aren't installed, installations of already installed versions & requests to install and remove a package at the same time.
2016-05-10edsp: support generic and solver-specific configsDavid Kalnischkies
The spec was slightly inconsistent if the preferences setting is available only as generic or specific setting & the code only supported the specific one, while for the strict-pinning was only generic… As the usual pattern for apt is to have both options we adapt the spec and code to support both as well. This also adds a purely informal "Solver" field so in case the request is saved in a file, we know to which solver the sent preferences apply. Closes: 823918
2016-05-10don't sent uninstallable rc-only versions via EDSPDavid Kalnischkies
Versions which are only available in dpkg/status aren't installable and apt doesn't pick them as candidate for this reason – for the same reason such packages shouldn't be sent to an external solver via EDSP. The packages are pinned to -1, but if the solver has strict pinning disabled it could end up picking this version anyhow – which is a request apt can not satisfy. Reported-By: Maximiliano Curia <maxy@debian.org> on IRC
2016-04-25edsp: ask policy engine for the pin of the version directlyDavid Kalnischkies
2016-02-16pass versioned provides to external solvers in EDSPDavid Kalnischkies
The EDSP output generated by apt didn't include the versioned provides information so that every provides looked like an unversioned one in the eyes of an external resolver.
2015-11-04sanify API to get 'the' candidate versionDavid Kalnischkies
This was discussed a while ago on #debian-apt and now that I see myself making this mistake lets bite the bullet and fix it in the easy way out version: Using a new name which fits with a similar named setter and deprecate the old method instead of 'hostily' changing API. Closes: #803471
2015-09-14do not discard new manual-bits while applying EDSP solutionsDavid Kalnischkies
In private-install.cc we call MarkInstall with FromUser=true, which sets the bit accordingly, but while applying the EDSP solution we call mark install on all packages with FromUser=false, so MarkInstall believes this install is an automatic one and sets it to auto – so that a new package which is explicitely installed via an external solver is marked as auto and is hence also up for garbage collection in a following call. Ideally MarkInstall wouldn't reset it, but the detection is hard to do without regressing in other cases – and ideally ideally MarkInstall wouldn't deal with the autobit at all – so we work around this on the calling side for now.
2015-09-14add Source-Version field for EDSPDavid Kalnischkies
The syntax of "Source" is different in EDSP compared to the the field of the same name in 'the rest' of Debian, so documented this accordingly and send the version as a new field.
2015-09-14implement dpkgs vision of interpreting pkg:<arch> dependenciesDavid Kalnischkies
How the Multi-Arch field and pkg:<arch> dependencies interact was discussed at DebConf15 in the "MultiArch BoF". dpkg and apt (among other tools like dose) had a different interpretation in certain scenarios which we resolved by agreeing on dpkg view – and this commit realizes this agreement in code. As was the case so far libapt sticks to the idea of trying to hide MultiArch as much as possible from individual frontends and instead translates it to good old SingleArch. There are certainly situations which can be improved in frontends if they know that MultiArch is upon them, but these are improvements – not necessary changes needed to unbreak a frontend. The implementation idea is simple: If we parse a dependency on foo:amd64 the dependency is formed on a package 'foo:amd64' of arch 'any'. This package is provided by package 'foo' of arch 'amd64', but not by 'foo' of arch 'i386'. Both of those foo packages provide each other through (assuming foo is M-A:foreign) to allow a dependency on 'foo' to be satisfied by either foo of amd64 or i386. Packages can also declare to provide 'foo:amd64' which is translated to providing 'foo:amd64:any' as well. This indirection over provides was chosen as the alternative would be to teach dependency resolvers how to deal with architecture specific dependencies – which violates the design idea of avoiding resolver changes, especially as architecture-specific dependencies are a cornercase with quite a few subtil rules. Handling it all over versioned provides as we already did for M-A in general seems much simpler as it just works for them. This switch to :any has actually a "surprising" benefit as well: Even frontends showing a package name via .Name() [which doesn't show the architecture] will display the "architecture" for dependencies in which it was explicitely requested, while we will not show the 'strange' :any arch in FullName(true) [= pretty-print] either. Before you had to specialcase these and by default you wouldn't get these details shown. The only identifiable disadvantage is that this complicates error reporting and handling. apt-get's ShowBroken has existing problems with virtual packages [it just shows the name without any reason], so that has to be worked on eventually. The other case is that detecting if a package is completely unknown or if it was at least referenced somewhere needs to acount for this "split" – not that it makes a practical difference which error is shown… but its one of the improvements possible.
2015-08-17Fix all the wrong removals of includes that iwyu got wrongMichael Vogt
Git-Dch: ignore
2015-08-17Cleanup includes after running iwyuMichael Vogt
2015-08-10hide implicit deps in apt-cache again by defaultDavid Kalnischkies
Before MultiArch implicits weren't a thing, so they were hidden by default by definition. Adding them for MultiArch solved many problems, but having no reliable way of detecting which dependency (and provides) is implicit or not causes problems everytime we want to output dependencies without confusing our observers with unneeded implementation details. The really notworthy point here is actually that we keep now a better record of how a dependency came to be so that we can later reason about it more easily, but that is hidden so deep down in the library internals that change is more the problems it solves than the change itself.
2015-08-10remove the compatibility markers for 4.13 abiDavid Kalnischkies
We aren't and we will not be really compatible again with the previous stable abi, so lets drop these markers (which never made it into a released version) for good as they have outlived their intend already. Git-Dch: Ignore
2015-06-16add d-pointer, virtual destructors and de-inline de/constructorsDavid Kalnischkies
To have a chance to keep the ABI for a while we need all three to team up. One of them missing and we might loose, so ensuring that they are available is a very tedious but needed task once in a while. Git-Dch: Ignore
2015-06-12store Release files data in the CacheDavid Kalnischkies
We used to read the Release file for each Packages file and store the data in the PackageFile struct even through potentially many Packages (and Translation-*) files could use the same data. The point of the exercise isn't the duplicated data through. Having the Release files as first-class citizens in the Cache allows us to properly track their state as well as allows us to use the information also for files which aren't in the cache, but where we know to which Release file they belong (Sources are an example for this). This modifies the pkgCache structs, especially the PackagesFile struct which depending on how libapt users access the data in these structs can mean huge breakage or no visible change. As a single data point: aptitude seems to be fine with this. Even if there is breakage it is trivial to fix in a backportable way while avoiding breakage for everyone would be a huge pain for us. Note that not all PackageFile structs have a corresponding ReleaseFile. In particular the dpkg/status file as well as *.deb files have not. As these have only a Archive property need, the Component property takes over this duty and the ReleaseFile remains zero. This is also the reason why it isn't needed nor particularily recommended to change from PackagesFile to ReleaseFile blindly. Sticking with the earlier is usually the better option.
2014-11-08guard ABI changes for SourcePkg/Ver in pkgCacheDavid Kalnischkies
Git-Dch: Ignore
2014-09-27store source name and version in binary cacheDavid Kalnischkies
Accessing the package records to acquire this information is pretty costly, so that information wasn't used so far in many places. The most noticeable user by far is EDSP at the moment, but there are ideas to change that which this commit tries to enable.
2014-06-18don't send pkg from an unknown architecture via EDSPDavid Kalnischkies
APT's cache can include packages from architectures dpkg has no knowledge about and can therefore not be installed for e.g. to allow easy lookups. There is no point in telling external solvers about them though and some of them might even be really talkative about ignoring them if we do.
2014-05-30support parsing EDSP requests Architecture{,s} stanzaDavid Kalnischkies
Adds also a small testcase for EDSP Git-Dch: Ignore
2014-05-30check exit status of external solversDavid Kalnischkies
Solvers are supposed to exit successfully even if they haven't found a solution, but a solver which fails drastically (like e.g. segfaults) should be detected and dealt with accordingly instead of ignored.
2014-05-04EDSP: add APT-Release field to Package stanzasStefano Zacchiroli
2014-05-04EDSP: add Source field to Package stanzasStefano Zacchiroli
2014-05-04EDSP: add Architecture(s) multi-arch fields to the Request stanzaStefano Zacchiroli
2014-05-04EDSP: bump protocol version to 0.5Stefano Zacchiroli
2014-03-13cleanup headers and especially #includes everywhereDavid Kalnischkies
Beside being a bit cleaner it hopefully also resolves oddball problems I have with high levels of parallel jobs. Git-Dch: Ignore Reported-By: iwyu (include-what-you-use)
2014-03-13warning: unused parameter ‘foo’ [-Wunused-parameter]David Kalnischkies
Reported-By: gcc -Wunused-parameter Git-Dch: Ignore
2013-03-20apt-pkg/edsp.cc: do not use stderr in WriteSolution at allMichael Vogt
2013-03-20apt-pkg/edsp.cc: do not spam stderr in WriteSolutionMichael Vogt