summaryrefslogtreecommitdiff
path: root/apt-pkg
AgeCommit message (Collapse)Author
2016-08-31protect only the latest same-source providers from autoremoveDavid Kalnischkies
Traditionally all providers are protected providing something as apt can't know which of them is actually really providing the functionality for the user ensuring that we don't propose the removal of used stuff, but that is of course also keeping stuff around which could be removed. That can cause the collection of multiple old providers until the provided package is itself no longer needed (e.g. out-of-tree kernel modules). We combat this by marking providers only from the newest source package version so that old providers built by older versions of the same source package can be garbage collected. (cherry picked from commit a0ed43f7323b9d7976ed0ba8d437a42e24af9eaf)
2016-08-31more explicit MarkRequired algorithm code (part 2)David Kalnischkies
As the previous commit, this shouldn't change behavior at all, but beside being more explicit and perhaps faster its also considerably shorter (granted, mostly by if0-block elimination). Gbp-Dch: Ignore (cherry picked from commit 5a3339db48479114a0e1e11ebc8d640eb3e49933)
2016-08-31more explicit MarkRequired algorithm codeDavid Kalnischkies
Piling everything in a single if statement always made my head wobble, but it hasn't even a benefit as the most common case of a package which isn't installed passes all of the old if and lands in the non-existent else-part of the inner if. So beside a subjective cleanup of what goes on this implementation should also be a bit faster. No change in behavior should be present. Gbp-Dch: Ignore (cherry picked from commit 769e9f3ea1cbe67d3b98e6db6c956abde2384868)
2016-08-31factor out Pkg/DepIterator prettyprinters into own headerDavid Kalnischkies
The old prettyprinters have only access to the struct they pretty print, which isn't enough usually as we want to know for a package also a bit of state information like which version is the candidate. We therefore need to pull the DepCache into context and hence use a temporary struct which is printed instead of the iterator itself. (cherry picked from commit 84573326f41dd09b914b8374548e7ee7c93d0439)
2016-08-31write auto-bits before calling dpkg & again after if neededDavid Kalnischkies
Writing first means that even in the event of a power-failure the autobit is saved for future processing instead of "forgotten" so that the package is treated as manually installed. In some cases we have to re-run the writing after dpkg is done through as dpkg can let packages disappear and in such cases apt will move autobits around (or in that case non-autobits) which we need to store. (cherry picked from commit 309f497b7280a45e3626493318adb6d39ba5c69b)
2016-08-31if reading of autobit state failed, let write failDavid Kalnischkies
If we can't read the old file we can't just move forward as that would discard potentially discard old data (especially other fields). We let it fail only after we are done writing the new file so a user has the chance to look into and merge the new data (which is otherwise discarded). (cherry picked from commit 520931867ee2fac8415a624204414d3b62550996)
2016-08-31don't do atomic overrides with failed filesDavid Kalnischkies
We deploy atomic renames for some files, but these renames also happen if something about the file failed which isn't really the point of the exercise… Closes: 828908 (cherry picked from commit fc5db01bb7d1546944200d197866b0b5c378f100)
2016-08-31cache: Bump minor version to 6Julian Andres Klode
Needed for the previous change (cherry picked from commit 33aa2752e7c7a6f0a01b191111aa35a5fe69cf20)
2016-08-31Fix buffer overflow in debListParser::VersionHash()Julian Andres Klode
If a package file is formatted in a way that that no space follows a deprecated "<", we would reformat it to "<=" and increase the length of the output by 1, which can break. Under normal circumstances with "<=" this should not be an issue. Closes: #828812 (cherry picked from commit b6e9756ca03ec887ef1d0bc8e38f63c29db7a365)
2016-06-22ensure filesize of deb is included in the hashes listDavid Kalnischkies
Filesize is a silly hash all by itself, but in combination with others it can be a strong opponent, so ensuring that it is in the list of hashes and hence checked by the normal course of action the acquire process takes is a good thing. (cherry picked from commit 5da51e0e2da3f055306562d38103b06a23d81719)
2016-06-20do not error if auto-detect-proxy cmd has no outputDavid Kalnischkies
Regression introduced in 8f858d560e3b7b475c623c4e242d1edce246025a. Commands are probably better of always having output through as the fall through to the generic proxy settings is likely not intended. As documenting and implementing this more consistently is kind of a regression through, it is split off into the next commit. Closes: 827713 (cherry picked from commit cad1877559f3e1703c3fea4d081978e1b4bb4a0e)
2016-06-10don't leak FD in AutoProxyDetect command return parsingDavid Kalnischkies
Just closing the fd would be enough, but while we are at it we can also use the Popen interface to have an easier time with this. (cherry picked from commit 8f858d560e3b7b475c623c4e242d1edce246025a)
2016-06-10don't leak an FD in lz4 (de)compressionDavid Kalnischkies
Seen first in #826783, but as this buglog also shows leaked uncompressed files as well we don't close it just yet. (cherry picked from commit 6f35be91c9e86e463bca7df6eadf05412c7b732c)
2016-06-10do not hang on piped input in PipedFileFdPrivateDavid Kalnischkies
This effects only compressors configured on the fly (rather then the inbuilt ones as they use a library). (cherry picked from commit bdc42211700ef0f6f40e4ef3f362e52d684d70fb)
2016-06-01prevent C++ locale number formatting in text APIsDavid Kalnischkies
Setting the C++ locale via std::locale::global(std::locale("")); which would otherwise default to the default C locale (aka: unaffected by setlocale) effects the formatting of numeric types in IO streams, which for output for humans is perfectly sensible, but breaks our many text interfaces used and parsed by us and others without expecting the numbers to be formatted. Closes: #825396 (cherry picked from commit b58e2c7c56b1416a343e81f9f80cb1f02c128e25)
2016-05-20fail instead of segfault on unreadable config filesDavid Kalnischkies
The report mentions "apt list --upgradable", but there are others which have inconsistent behavior ranging from segfaulting to doing something with the partial (and hence incomplete) data. We had a recent report about sources.list (#818628), this one mentions prefences, the obvious next step is conf files… so the testcase is adapted to check for all three in file and directory versions and run a bunch of commands each time which should all have more or less the same behavior in such a case (aka error out). Closes: 824503 (cherry picked from commit fdf9eef4d96a18d0167708499c993e1174251e88)
2016-05-11respect user pinning in M-A:same version (un)screwingDavid Kalnischkies
Using Pkg.CandVersion() here is wrong as its implementation will return a candidate based just on the default policy settings ignoring user preferences and otherwise set candidates (aka: it sidesteps the pkgDepCache). This causes M-A:same libraries to be detected as screwed even through they aren't, so that they end up being kept back. Reported-By: Felipe Sateler on IRC
2016-05-10update: Run Post-Invoke-Success if not all sources failedJulian Andres Klode
Failures can happen and APT regardless will do a partial cache update anyway. Because APT ensures that the list directory is in a sane state, it makes sense to also call success hooks if success was only partial - otherwise it loses sync with APT. Most importantly, this causes the appstream cache to be empty, see launchpad bug #1562733. This is somewhat overly optimistic though: As soon as any repository has nonexisting optional files, the missing optional files are also treated as success, which means a single broken repository without an InRelease file still runs Success hooks, even though it really should not. (cherry picked from commit 35664152e47a1d4d712fd52e0f0a2dc8ed359d32)
2016-05-10don't sent uninstallable rc-only versions via EDSPDavid Kalnischkies
Versions which are only available in dpkg/status aren't installable and apt doesn't pick them as candidate for this reason – for the same reason such packages shouldn't be sent to an external solver via EDSP. The packages are pinned to -1, but if the solver has strict pinning disabled it could end up picking this version anyhow – which is a request apt can not satisfy. Reported-By: Maximiliano Curia <maxy@debian.org> on IRC (cherry picked from commit 33190fe3d3c200dcd417cd336f9db11f5f4408d5)
2016-05-10allow redirection for items without a space in the desc againDavid Kalnischkies
Broken in a4b8112b19763cbd2c12b81d55bc7d43a591d610. If an item has a description which includes no space and is redirected to another mirror the code which wants to rewrite the description expects a space in there, but can't find it and the unguarded substr command on the string will fail with an exception thrown… Guarding it properly and everything is fine. (cherry picked from commit 84ac6edfabe1c92d67e8d441e04216ad33c89165)
2016-05-10don't show NO_PUBKEY warning if repo is signed by another keyDavid Kalnischkies
Daniel Kahn Gillmor highlights in the bugreport that security isn't improving by having the user import additional keys – especially as importing keys securely is hard. The bugreport was initially about dropping the warning to a notice, but in given the previously mentioned observation and the fact that we weren't printing a warning (or a notice) for expired or revoked keys providing a signature we drop it completely as the code to display a message if this was the only key is in another path – and is considered critical. Closes: 618445 (Backported from commit fb7b11ebb852fa255053ecab605bc9cfe9de0603)
2016-05-10deb822: Restore support for <multivalue>-{Add,Remove}James McCoy
Redesign of multivalue options in 463c8d801595ce5ac94d7c032264820be7434232 caused the parser to look for <multivalue>{Add,Remove} (no hyphen) instead of the expected <multivalue>-{Add,Remove}. (cherry picked from commit f5585106d61b381c9dcf8f1dd48c742dc68f6c81)
2016-04-14ensure outdated files are dropped without lists-cleanupDavid Kalnischkies
Tested via (newly) empty index files, but effects also files dropped from the repository or an otherwise changed repository config.
2016-04-14silently skip acquire of empty index filesDavid Kalnischkies
There is just no point in taking the time to acquire empty files – especially as it will be tiny non-empty compressed files usually.
2016-04-14fix Alt-Filename handling of file methodDavid Kalnischkies
A silly of-by-one error in the stripping of the extension to check for the uncompressed filename broken in an attempt to support all compressions in commit a09f6eb8fc67cd2d836019f448f18580396185e5. Fixing this highlights also mistakes in the handling of the Alt-Filename in libapt which would cause apt to remove the file from the repository (if root has the needed rights – aka the disk isn't readonly or similar)
2016-04-13recheck Pre-Depends satisfaction in SmartConfigureDavid Kalnischkies
Regression introduced in commit 590f1923121815b36ef889033c1c416a23cbe9a2 (2011!) causing apt to not check if Pre-Depends are satisfied before calling --configure. This managed to hide so perfectly well for years as Pre-Depends aren't that common, apt prefers upgrading these packages first and checks for satisfaction is already in SmartUnpack, so there is only a small window of oppertunity to break a pre-dependency relation (usually with an unpack). Verified by logchecking with two provided status files in the buglog. I would have liked to write a test, but I wasn't able to reach the needed complexity to get apt to fail – but the change is small and reasonable, so what could possible go wrong™, right? LP: #1569099
2016-04-13detect compressed status files on extension againDavid Kalnischkies
It handy to be able to point apt at reading a compressed dpkg/status file in debugging cases, which worked pre-1.1 but somewhere down the line in the massive refactoring. Restoring this behavior in a central place for all realfile index files instead of just for the status file. (This has no effect on index files acquired from an archive – those are handled by different classes and support compressed files just fine)
2016-04-13do not require non-broken systems in 'upgrade'David Kalnischkies
There is a good chance that the attempt will fail, but if a user mentions certain packages explicitly on the commandline there is a chance that this will consist of a broken system which is resolved by upgrading more packages then just the mentioned. This limitation was not effecting external resolvers.
2016-04-07stop handling items in doomed transactionsDavid Kalnischkies
With the previous commit we track the state of transactions, so we can now use our knowledge to avoid processing data for a transaction which was already closed (via an abort in this case). This is needed as multiple independent processes are interacting in the process, so there isn't a simple immediate full-engine stop and it would also be bad to teach each and every item how to check if its manager has failed subordinate and what to do in that case. In the pdiff case, which deals (potentially) with many items during its lifetime e.g. a hashsum mismatch in another file can abort the transaction the file we try to patch via pdiff belongs to. This causes some of the items (which are already done) to be aborted with it, but items still in the process of acquisition continue in the processing and will later try to use all the items together failing in strange ways as cleanup already happened. The chosen solution is to dry up the communication channels instead by ignoring new requests for data acquisition, canceling requests which are not assigned to a queue and not calling Done/Failed on items anymore. This means that e.g. already started or pending (e.g. pipelined) downloads aren't stopped and continue as normal for now, but they remain in partial/ and aren't processed further so the next update command will pick them up and put them to good use while the current process fails updating (for this transaction group) in an orderly fashion. Closes: 817240 Thanks: Barr Detwix & Vincent Lefevre for log files
2016-04-07ensure transaction states are changed only onceDavid Kalnischkies
We want to keep track of the state of a transaction overall to base future decisions on it, but as a pre-requirement we have to make sure that a transaction isn't commited twice (which happened if the download of InRelease failed and Release takes over). It also happened to create empty commits after a transaction was already aborted in cases in which the Release files were rejected. This isn't effecting security at the moment, but to ensure this isn't happening again and can never be bad a bunch of fatal error messages are added to make regressions on this front visible.
2016-04-03don't leak on error in listparser creationDavid Kalnischkies
Git-Dch: Ignore Reported-By: gcc -fsanitize=address
2016-04-03use buffered writing for InRelease splittingDavid Kalnischkies
Hardly noticeable, but given that we have the option to easily enable it, lets enable it as every newline in the message is written individually by the code.
2016-04-03consistently add APT_OVERRIDE as method markerDavid Kalnischkies
Some methods had it missing, some used the keyword directly, which isn't a problem as it is a cc file, but for consistency lets stick to our macro for now. Git-Dch: Ignore
2016-03-28Allow lowering trust level of a hash via configJulian Andres Klode
Introduces APT::Hashes::<NAME> with entries Untrusted and Weak which can be set to true to cause the hash to be treated as untrusted and/or weak.
2016-03-27Do not mark packages for keep that we want to removeJulian Andres Klode
If the package is marked for removal, keep it marked for removal and do not mark it for keep. If we mark it for keep, we some how later get to a different stage where it is marked for unpack instead of removal. In the example in the bug report, we would get a: SmartUnPack maas-region-controller-min:amd64 (replace version 2.0.0~alpha3+bzr4810-0ubuntu1 with Segmentation fault maas-region-controller-min:amd64 was marked for removal, but we changed it to keep and somehow it thinks that this is to be replaced now instead of removed (probably because the InstallVer != CandidateVer [with InstallVer = 0]). This fixes a regression introduced in release 1.2.7, commit: 0390edd5452b081f8efcf412f96d535a1d959457 Reported-by: LaMont Jones on IRC LP: #1562402
2016-03-25drop confusing comma from no strong hash messageDavid Kalnischkies
2016-03-24Set hidden visibility for StringViewJulian Andres Klode
This avoids templates using StringView to be exported, such as std::vector<StringView*>::emplace_back(). Gbp-Dch: ignore
2016-03-19refactor loading of previous release fileDavid Kalnischkies
There is really no need to have the same code three times. Git-Dch: Ignore
2016-03-19cachefile: Only set members that were initialized successfullyJulian Andres Klode
Otherwise, things will just start failing later down the stack, because (a) the lazy getters do not check if building was successful and (b) any further getter call would return the invalid object anyway. Also initialize VS in pkgCache to nullptr by default. Closes: #818628
2016-03-16do not strip epochs from state version stringsDavid Kalnischkies
The epoch stripping in this code is done since day one, but in other places we show a version epochs are not stripped. If epochs are present in packages they tend to be an important information which we can't just drop and especially can't drop "sometimes" as that confuses users and tools alike – so even if removing code in use for (close to) 18 years feels wrong, it is probably the right choice for consistency. Closes: 818162
2016-03-16Use descriptive URIs in 104 Warning messagesJulian Andres Klode
This makes the new GPG related warnings much nicer to read, for example, the second one here replaces the first one: W: gpgv:/var/lib/apt/lists/example.com_dists_stable_InRelease: Weak ... W: http://example.com/dists/stable/InRelease: Weak ...
2016-03-16Report non-transient errors as errors, not as warningsJulian Andres Klode
This makes it easier to understand what really is an error and what not.
2016-03-16Get accurate progress reporting in apt update againMichael Vogt
For the non-pdiff case, we have can have accurate progress reporting because after fetching the {,In}Release files we know how many IndexFiles will be fetched and what size they have. Therefore init the filesize early (in pkgAcqIndex::Init) and ensure that in Acquire::Pulse() looks at already downloaded bits when calculating the progress in Acquire::Pulse. Also improve debug output of Debug::acquire::progress
2016-03-15Fix bug where the problemresolve can put a pkg into a heisenstateMichael Vogt
The problemresolver will set the candidate version for pkg P back to the current version if it encounters an impossible to satisfy critical dependency on P. However it did not set the State of the package back as well which lead to a situation where P is neither in Keep,Install,Upgrade,Delete state. Note that this can not be tested via the traditional sh based framework. I added a python-apt based test for this. LP: #1550741 [jak@debian.org: Make the test not fail if apt_pkg cannot be imported]
2016-03-15apt-pkg/acquire-worker.cc: Introduce 104 Warning messageJulian Andres Klode
This can be used by workers to send warnings to the main program. The messages will be passed to _error->Warning() by APT with the URI prepended. We are not going to make that really public now, as the interface might change a bit.
2016-03-14enforce verify of filesize in 'apt-get source'David Kalnischkies
The structure we parse the data into has a dedicated size field, but it tends to be easier to handle it as a (very weak) checksum.
2016-03-14don't use Desc.URI to calculate .diff/Index filenamesDavid Kalnischkies
The URI descibing an item can change via mirrors/redirectors which causes the .diff/Index files to get the wrong names in storage. Git-Dch: Ignore
2016-03-14streamline dpkgpm cleanup-handlingDavid Kalnischkies
The (unlikely) waitpid failure case should fallthrough the code just like the other failures (and successes) instead of taking a shortcut avoiding all the cleanup (progress) and finishing touches (log, state). This also delays the cleanup of the progress until apt is really done with everything and "just" has the post-invokes left to do, so the period of 'apt looks finished as it stopped the progress' and 'apt really finished as I have the shell-prompt back' is shorter even if there is no progress reported anymore, so the bar lingers at 100%… Ideally even the post-invokes would be covered by progress, but they can have their own output and dealing with that could be hard. Git-Dch: Ignore
2016-03-14flush line-clearing on progress stop before post-invokeDavid Kalnischkies
All other interactions with std::cout are flushed directly, just in the stop case we hadn't done it – no problem expect if there is still output coming after apt is done like in the case of a post-invoke script producing output. Closes: 793672
2016-03-14require $(HASH)-Download field in .diff/Index filesDavid Kalnischkies
Now that we ignore SHA1-only files it makes sense to require also the provision of hashes for the compressed patches as this was introduced in the same patchset as support for non-SHA1 hashes in the file itself in dak and adding support in other archive creators (if they support pdiffs at all) will likely be in the same batch. The reason for the change itself is simple: If you are 'scared' enough about the security of SHA1, you shouldn't uncompress a file you haven't verified at all – after all, it could be exploiting a bug or a zip bomb.