summaryrefslogtreecommitdiff
path: root/apt-pkg
AgeCommit message (Collapse)Author
2016-06-10don't leak EDSP solver output fdDavid Kalnischkies
2016-06-10don't leak an FD in lz4 (de)compressionDavid Kalnischkies
Seen first in #826783, but as this buglog also shows leaked uncompressed files as well we don't close it just yet.
2016-06-10do not hang on piped input in PipedFileFdPrivateDavid Kalnischkies
This effects only compressors configured on the fly (rather then the inbuilt ones as they use a library).
2016-06-09remove racy_pselect fallbackDavid Kalnischkies
The comment says it should have been removed with Lenny+1 which is a small while ago already, so it seems like a good time now… And as this is a cleanup commit it also gets right of spurious whitespace at the end of lines, adds missing fold markers and similar busy work. Git-Dch: Ignore
2016-06-08drop Dpkg::MaxArgs in favor of Dpkg::MaxArgsBytesDavid Kalnischkies
We had an old FIXME saying that it is probably pointless to do this if we limit by length of the commandline already and I completely agree. The splitting is bad enough if it must be done, so we should only do it if we have to (as in absolute length of commandline) and, but that is just a remark, it is unlikely that we ever have/had a call triggering this as the default value was ~32000 items…
2016-06-08don't explicitly configure the last round of packagesDavid Kalnischkies
We end our operation by calling "dpkg --configure -a", so instead of running a (big) configure run with all packages mentioned explicitly before this, we simply skip them and let them be handled by this call implicitly. There isn't really an observeable gain to be had here from a speed point, but it helps in avoiding an (uncommon) problem of having a too long commandline passed to dpkg, which we would split up (probably incorrectly).
2016-06-08edsp: drop privileges before executing solversDavid Kalnischkies
Most (if not all) solvers should be able to run perfectly fine without root privileges as they get the entire state they are supposed to work on via stdin and do not perform any action directly, but just pass suggestions on via stdout. The new default is to run them all as _apt hence, but each solver can configure another user if it chooses/must. The security benefits are minimal at best, but it helps preventing silly mistakes (see 35f3ed061f10a25a3fb28bc988fddbb976344c4d) and that is always good. Note that our 'apt' and 'dump' solver already dropped privileges if they had them.
2016-06-08edsp: if logging is requested, do it for internal, tooDavid Kalnischkies
Create and log the EDSP(like) request even if we use the internal resolver.
2016-06-08edsp: if internal is used, keep this decisionDavid Kalnischkies
It wasn't noticeable before, but now with the (optional) logging it can be observed that we decide in the internal path two times if an internal or external solver should be used (and hence with logging, it is attempted twice), so if we are in the internal path call the internal resolver directly, which means those internal methods need to be public – but we can hide them based on the symbol at least.
2016-06-08edsp: optionally store a compressed copy of the last scenarioDavid Kalnischkies
For bugreports and co it could be handy to have the scenario and all the settings used in it around later for inspection for EDSP like protocols. EDSP might not be the most interesting as the user can still interrupt the process before the solution is applied and users tend to have an opinion on the "rightness" of a solution, so it is disabled by default.
2016-06-06edsp: prevent it from trying to write a cacheDavid Kalnischkies
EDSP(-like) protocols are one-shot processes working on data which exists only as long as they run (as they get feed via a pipe), so trying to write a cache for it is pretty pointless, especially as it will usually fail as the cache files tend to be owned by root, but the process is run as a unpriviledged user (either _apt if called by root, otherwise the user of the caller). So this was in fact only observeable with our testcases which run as non-root and the worst which happens is that a valid cache is overridden with an invalid one which the next run will detect and not use. Git-Dch: Ignore
2016-06-06get right of the hardcoded string-length comparesDavid Kalnischkies
With have better ways to compare, manipulate and work with strings, so use it instead of counting string length by hand with is a wonder it hasn't failed yet. Ignoreable from a changelog perspective as there is no behaviour change. Git-Dch: Ignore
2016-06-05refactor EDSP classes for better internal reuseDavid Kalnischkies
The classes are all marked as hidden, so changing them is no problem ABI wise and will help with introducing protocols similar to EDSP. The change has no observeable behavior difference, its just code juggling. Git-Dch: Ignore
2016-06-04edsp: use a stanza based interface for solution writingDavid Kalnischkies
EDSP had a WriteSolution method to write out the entire solution based on the inspection of a given pkgDepCache, but that is rather inflexible both for EDSP itself and for other EDSP like-protocols. It seems better to use a smaller scope in printing just a single stanza based on a given version as there is more reuse potential.
2016-06-04edsp: use an ID mapping for the internal solverDavid Kalnischkies
Currently an EDSP solver gets send basically all versions which means the absolute count is the same, but that might not be true forever (and with the skipping of rc-only versions it kinda is already) and even if it were true, segfaulting on bad input seems wrong.
2016-05-28use de-localed std::put_time instead rolling our ownDavid Kalnischkies
2016-05-28accept only the expected UTC timezones in date parsingDavid Kalnischkies
HTTP/1.1 hardcodes GMT (RFC 7231 §7.1.1.1) and what is good enough for the internet must be good enough for us™ as we reuse the implementation internally to parse (most) dates we encounter in various places like the Release files with their Date and Valid-Until header fields. Implementing a fully timezone aware parser just feels too hard for no effective benefit as it would take 5+ years (= until LTS's are out of fashion) until a repository could use non-UTC dates and expect it to work. Not counting non-apt implementations which might or might not only want to encounter UTC here as well. As a bonus, this eliminates the use of an instance of setlocale in libapt. Closes: 819697
2016-05-27prevent C++ locale number formatting in text APIsDavid Kalnischkies
Setting the C++ locale via std::locale::global(std::locale("")); which would otherwise default to the default C locale (aka: unaffected by setlocale) effects the formatting of numeric types in IO streams, which for output for humans is perfectly sensible, but breaks our many text interfaces used and parsed by us and others without expecting the numbers to be formatted. Closes: #825396
2016-05-27fix and document on the fly compressor configDavid Kalnischkies
libapt allows to configure compressors to be used by its system via configuration implemented in 03bef78461c6f443187b60799402624326843396, but that was never really documented and also only partly working, which also explains why the tests weren't using it…
2016-05-24fix two typos in untranslated errors of libapt-pkgDavid Kalnischkies
Reported-By: lintian: spelling-error-in-binary Git-Dch: Ignore
2016-05-20refactor EDSP code into EDSP and EDSP-like partsDavid Kalnischkies
No real code change, just moving code around heavily to decouple the EDSP specific parts from those we can reuse for EDSP-like protocols. Git-Dch: Ignore
2016-05-20edsp: use a limited scenario based on bool-arrayDavid Kalnischkies
Its more space and runtime efficient to use a boolean set instead of a CacheSet-based implementation. Git-Dch: Ignore
2016-05-20edsp: add Forbid-{New-Install,Remove} and Upgrade-AllDavid Kalnischkies
This allows to differentiate properly between 'apt-get upgrade', 'apt upgrade' and 'apt full-upgrade'.
2016-05-20avoid triggering gcc's -Wunsafe-loop-optimizations in EDSPDavid Kalnischkies
apt/apt-pkg/edsp.cc: In function ‘bool EDSP::WriteLimitedScenario(pkgDepCache&, FILE*, const PackageSet&, OpProgress*)’: apt/apt-pkg/edsp.cc:245:56: warning: cannot optimize loop, the loop counter may overflow [-Wunsafe-loop-optimizations] std::string dependencies[pkgCache::Dep::Enhances + 1]; ^ Using a std::array to silence gcc as well as as a code improvement feels right here. Git-Dch: Ignore
2016-05-20edsp: try harder to not generate unneeded error messagesDavid Kalnischkies
The &= introduced in the EDSP-FileFd conversion isn't working to full satisfaction for multiple && clauses as the && has a higher binding than &= has, so that the methods were called even through they shouldn't have because of previous errors. Using variadic functions we can solve this in a slightly cleaner way bringing down the amount of 'broken pipe' errors for the error case of the dump resolver substantially. Git-Dch: Ignore
2016-05-20convert EDSP to be based on FileFd instead of FILE*David Kalnischkies
I doubt there is any non-src:apt usage of these interfaces.
2016-05-20fail instead of segfault on unreadable config filesDavid Kalnischkies
The report mentions "apt list --upgradable", but there are others which have inconsistent behavior ranging from segfaulting to doing something with the partial (and hence incomplete) data. We had a recent report about sources.list (#818628), this one mentions prefences, the obvious next step is conf files… so the testcase is adapted to check for all three in file and directory versions and run a bunch of commands each time which should all have more or less the same behavior in such a case (aka error out). Closes: 824503
2016-05-15Normalize Signed-By values by removing trailing commas everywhereJulian Andres Klode
This fixes comparisons where either the stored or the input string have a trailing comma.
2016-05-15Add conflicting Signed-By values to error messageJulian Andres Klode
This hopefully makes debugging things easier.
2016-05-12edsp: warn if unexpected stanzas appear in the solutionDavid Kalnischkies
Unexpected are for examples removal requests for versions which aren't installed, installations of already installed versions & requests to install and remove a package at the same time.
2016-05-10edsp: support generic and solver-specific configsDavid Kalnischkies
The spec was slightly inconsistent if the preferences setting is available only as generic or specific setting & the code only supported the specific one, while for the strict-pinning was only generic… As the usual pattern for apt is to have both options we adapt the spec and code to support both as well. This also adds a purely informal "Solver" field so in case the request is saved in a file, we know to which solver the sent preferences apply. Closes: 823918
2016-05-10update: Run Post-Invoke-Success if not all sources failedJulian Andres Klode
Failures can happen and APT regardless will do a partial cache update anyway. Because APT ensures that the list directory is in a sane state, it makes sense to also call success hooks if success was only partial - otherwise it loses sync with APT. Most importantly, this causes the appstream cache to be empty, see launchpad bug #1562733. This is somewhat overly optimistic though: As soon as any repository has nonexisting optional files, the missing optional files are also treated as success, which means a single broken repository without an InRelease file still runs Success hooks, even though it really should not.
2016-05-10Strip trailing commas for created signed-by fingerprint listsJulian Andres Klode
This prevented some sources.list entries from working, an example of which can be found in the test.
2016-05-10Merge pull request julian-klode/apt#3 from adrian17/masterJulian Andres Klode
Improve GetLocalitySortedVersionSet, speeds up apt search by 30%
2016-05-10don't sent uninstallable rc-only versions via EDSPDavid Kalnischkies
Versions which are only available in dpkg/status aren't installable and apt doesn't pick them as candidate for this reason – for the same reason such packages shouldn't be sent to an external solver via EDSP. The packages are pinned to -1, but if the solver has strict pinning disabled it could end up picking this version anyhow – which is a request apt can not satisfy. Reported-By: Maximiliano Curia <maxy@debian.org> on IRC
2016-05-08implement Identifier field for IndexTargetsDavid Kalnischkies
A frontend like apt-file is only interested in a specific set of files and selects those easily via "Created-By". If it supports two locations for those files through it would need to select both and a user would need to know that implementation detail for sources.list configuration. The "Identifier" field is hence introduced which by default has the same value as "Created-By", but can be freely configured – especially it can be used to give two indexes the same identifier.
2016-05-08implement Fallback-Of for IndexTargetsDavid Kalnischkies
Sometimes index files are in different locations in a repository as it is currently the case for Contents files which are per-component in Debian, but aren't in Ubuntu. This has historic reasons and is perhaps changed soon, but such cases of transitions can always happen in the future again, so we should prepare: Introduced is a new field declaring that the current item should only be downloaded if the mentioned item wasn't allowing for transitions without a flagday in clients and archives. This isn't implemented 'simpler' with multiple MetaKeys as items (could) change their descriptions and perhaps also other configuration bits with their location.
2016-05-07download arch:all also for NATIVE_ARCHITECTURE indextargetsDavid Kalnischkies
It looks a bit strange on the outside to have multiple "native architecture", but all is considered an implementation detail and e.g. packages of arch:all are in dependency resolution equal to native packages.
2016-05-07don't construct MetaIndex acquire items with IndexTargetsDavid Kalnischkies
We don't have to initialize the Release files with a set of IndexTargets to acquire, but instead wait for the Release file to be acquired and only then ask which IndexTargets to get. Git-Dch: Ignore
2016-05-07delay progress until Release files are downloadedDavid Kalnischkies
Progress reporting used an "upper bound" on files we might get, expect that this wasn't correct in case pdiff entered the picture. So instead of calculating a value which is perhaps incorrect, we just accept that we can't tell how many files we are going to download and just keep at 0% until we know. Additionally, if we have pdiffs we wait until we got these (sub)index files, too. That could all be done better by downloading all Release files first and planing with them in hand accordingly, but one step at a time.
2016-05-07TransactionManager can never be a nullptrDavid Kalnischkies
The code naturally evolved from a TransactionManager optional to a required setup which resulted in various places doing unneeded checks suggesting a more complicated setup than is actually needed. Git-Dch: Ignore
2016-05-07fix same-mirror redirection for Release{,.gpg} pairDavid Kalnischkies
Commit 9b8034a9fd40b4d05075fda719e61f6eb4c45678 just deals with InRelease properly and generates broken URIs in case the mirror (or the achieve really) has no InRelease file. [As this was in no released version no need to clutter changelog with a fix notice.] Git-Dch: Ignore
2016-05-03allow redirection for items without a space in the desc againDavid Kalnischkies
Broken in a4b8112b19763cbd2c12b81d55bc7d43a591d610. If an item has a description which includes no space and is redirected to another mirror the code which wants to rewrite the description expects a space in there, but can't find it and the unguarded substr command on the string will fail with an exception thrown… Guarding it properly and everything is fine.
2016-05-03let DPKG_COLORS default to our APT::Color settingDavid Kalnischkies
dpkg can optionally colorize its output since 1.18.5. Currently this defaults to 'never', but it will eventually be 'auto'. It seems reasonable to assume that a user who has enabled/disabled colors in apt will want to have dpkg have the same state regarding color usage. This isn't overriding explicit settings by the user, so in case a user feels strongly about it one way or the other there are options.
2016-05-03remove 100-levels config nesting limitDavid Kalnischkies
The actual reason for this commit isn't the limit – there isn't much point in using that much nesting – its in shutting up gcc mostly: apt/apt-pkg/contrib/configuration.cc: In function ‘bool ReadConfigFile(Configuration&, const string&, const bool&, const unsigned int&)’: apt/apt-pkg/contrib/configuration.cc:686:20: warning: cannot optimize loop, the loop counter may overflow [-Wunsafe-loop-optimizations] string Stack[100]; ^ by replacing this with C++s handy std::stack container (adapter). Also cleans some whitespace noise from the file in the process.
2016-05-01Don't copy strings in Startswith, EndswithAdrian Wielgosik
2016-05-01support Signed-By in Release files as a sort of HPKPDavid Kalnischkies
Users have the option since apt >= 1.1 to enforce that a Release file is signed with specific key(s) either via keyring filename or fingerprints. This commit adds an entry with the same name and value (except that it doesn't accept filenames for obvious reasons) to the Release file so that the repository owner can set a default value for this setting effecting the *next* Release file, not the current one, which provides a functionality similar "HTTP Public Key Pinning". The pinning is in effect as long as the (then old) Release file is considered valid, but it is also ignored if the Release file has no Valid-Until at all.
2016-05-01support multiple fingerprints in signed-byDavid Kalnischkies
A keyring file can include multiple keys, so its only fair for transitions and such to support multiple fingerprints as well.
2016-05-01don't show NO_PUBKEY warning if repo is signed by another keyDavid Kalnischkies
Daniel Kahn Gillmor highlights in the bugreport that security isn't improving by having the user import additional keys – especially as importing keys securely is hard. The bugreport was initially about dropping the warning to a notice, but in given the previously mentioned observation and the fact that we weren't printing a warning (or a notice) for expired or revoked keys providing a signature we drop it completely as the code to display a message if this was the only key is in another path – and is considered critical. Closes: 618445
2016-05-01show StateCache flags in Pkg debug prettyprintDavid Kalnischkies
This basically introduces ~33 flags in the output, but a package can have only ~11 of them displayed at the same time. There is quiet a bit of duplication also (an uninstalled package is by definition a newinstall if its getting installed), but as this is debug output we are better of showing them all in case one of them isn't set in a way it is supposed to be set. Git-Dch: Ignore