summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2016-05-08gpgv: show always webportal error on NODATADavid Kalnischkies
gpg doesn't give use a UID on NODATA, which we were "expecting" (but not using for anything), but just an error number. Instead of collecting these as badsigners which will trigger a "invald signature" error with remarks like "NODATA 1" we instead adapt a message similar to the NODATA error of a clearsigned file (which is actually not reached anymore as we split them up, which fails with a NOSPLIT error, which uses the same general error message). In other words: Not a security relevant change, just a user experience improvement as we now point them to the most likely cause of the problem instead of saying "invalid signature" which would point them in the direction of the archive being broken (for everyone) instead. Closes: 823746
2016-05-08implement Identifier field for IndexTargetsDavid Kalnischkies
A frontend like apt-file is only interested in a specific set of files and selects those easily via "Created-By". If it supports two locations for those files through it would need to select both and a user would need to know that implementation detail for sources.list configuration. The "Identifier" field is hence introduced which by default has the same value as "Created-By", but can be freely configured – especially it can be used to give two indexes the same identifier.
2016-05-08implement Fallback-Of for IndexTargetsDavid Kalnischkies
Sometimes index files are in different locations in a repository as it is currently the case for Contents files which are per-component in Debian, but aren't in Ubuntu. This has historic reasons and is perhaps changed soon, but such cases of transitions can always happen in the future again, so we should prepare: Introduced is a new field declaring that the current item should only be downloaded if the mentioned item wasn't allowing for transitions without a flagday in clients and archives. This isn't implemented 'simpler' with multiple MetaKeys as items (could) change their descriptions and perhaps also other configuration bits with their location.
2016-05-07download arch:all also for NATIVE_ARCHITECTURE indextargetsDavid Kalnischkies
It looks a bit strange on the outside to have multiple "native architecture", but all is considered an implementation detail and e.g. packages of arch:all are in dependency resolution equal to native packages.
2016-05-07don't construct MetaIndex acquire items with IndexTargetsDavid Kalnischkies
We don't have to initialize the Release files with a set of IndexTargets to acquire, but instead wait for the Release file to be acquired and only then ask which IndexTargets to get. Git-Dch: Ignore
2016-05-07delay progress until Release files are downloadedDavid Kalnischkies
Progress reporting used an "upper bound" on files we might get, expect that this wasn't correct in case pdiff entered the picture. So instead of calculating a value which is perhaps incorrect, we just accept that we can't tell how many files we are going to download and just keep at 0% until we know. Additionally, if we have pdiffs we wait until we got these (sub)index files, too. That could all be done better by downloading all Release files first and planing with them in hand accordingly, but one step at a time.
2016-05-07TransactionManager can never be a nullptrDavid Kalnischkies
The code naturally evolved from a TransactionManager optional to a required setup which resulted in various places doing unneeded checks suggesting a more complicated setup than is actually needed. Git-Dch: Ignore
2016-05-07fix same-mirror redirection for Release{,.gpg} pairDavid Kalnischkies
Commit 9b8034a9fd40b4d05075fda719e61f6eb4c45678 just deals with InRelease properly and generates broken URIs in case the mirror (or the achieve really) has no InRelease file. [As this was in no released version no need to clutter changelog with a fix notice.] Git-Dch: Ignore
2016-05-04tests: disable generation of Release.gpg by defaultDavid Kalnischkies
Most tests just need a signed repository and don't care if it signed by an InRelease file or a Release.gpg file, so we can save some time by just generating one of them by default. Sounds like not much, but quickly adds up to a few seconds with the amount of tests we have accumulated by now. Git-Dch: Ignore
2016-05-04tests: allow to disable generation of InRelease/Release.gpg fileDavid Kalnischkies
If the test just signs release files to throw away one of them to test the other, we can just as well save the time and not create it. Git-Dch: Ignore
2016-05-04test: fix permission issue if run as rootDavid Kalnischkies
Always those silly mistakes. Do what I mean, not what I said… Reported-By: Travis Git-Dch: Ignore
2016-05-03allow redirection for items without a space in the desc againDavid Kalnischkies
Broken in a4b8112b19763cbd2c12b81d55bc7d43a591d610. If an item has a description which includes no space and is redirected to another mirror the code which wants to rewrite the description expects a space in there, but can't find it and the unguarded substr command on the string will fail with an exception thrown… Guarding it properly and everything is fine.
2016-05-03let DPKG_COLORS default to our APT::Color settingDavid Kalnischkies
dpkg can optionally colorize its output since 1.18.5. Currently this defaults to 'never', but it will eventually be 'auto'. It seems reasonable to assume that a user who has enabled/disabled colors in apt will want to have dpkg have the same state regarding color usage. This isn't overriding explicit settings by the user, so in case a user feels strongly about it one way or the other there are options.
2016-05-03remove 100-levels config nesting limitDavid Kalnischkies
The actual reason for this commit isn't the limit – there isn't much point in using that much nesting – its in shutting up gcc mostly: apt/apt-pkg/contrib/configuration.cc: In function ‘bool ReadConfigFile(Configuration&, const string&, const bool&, const unsigned int&)’: apt/apt-pkg/contrib/configuration.cc:686:20: warning: cannot optimize loop, the loop counter may overflow [-Wunsafe-loop-optimizations] string Stack[100]; ^ by replacing this with C++s handy std::stack container (adapter). Also cleans some whitespace noise from the file in the process.
2016-05-01apt-key: add \n to dpkg-query --show --showformatCarsten Hey
Guarding against 'broken' greps not dealing with non-text inputs "just in case" by making the input text with a proper newline. [commit message by David Kalnischkies] Reported-On: IRC Git-Dch: Ignore
2016-05-01warn if apt-key is run unconditionally in maintainerscriptDavid Kalnischkies
We want to stop hard-depending on gnupg and for this it is essential that apt-key isn't used in any critical execution path, which maintainerscript are. Especially as it is likely that these script call apt-key either only for (potentially now outdated cleanup) or still not use the much simpler trusted.gpg.d infrastructure.
2016-05-01move gnupg|gnupg2 from apt Depends to RecommendsDavid Kalnischkies
apt doesn't need gnupg in its main execution paths to function, especially the Release file verification is done with gpgv only. It is only used by apt-key for advanced key management functionality most user will never use nor need. The intend is to demote it eventually to Suggests, but we opt here for a staged downgrade as there are still third-party repositories out there which require apt-key functionality without depending on gnupg (or apt for that matter).
2016-05-01ftparchive: Support writing Signed-By fieldsJulian Andres Klode
2016-05-01bugscript: include all configuration fragment filesDavid Kalnischkies
Closes: 820861
2016-05-01support Signed-By in Release files as a sort of HPKPDavid Kalnischkies
Users have the option since apt >= 1.1 to enforce that a Release file is signed with specific key(s) either via keyring filename or fingerprints. This commit adds an entry with the same name and value (except that it doesn't accept filenames for obvious reasons) to the Release file so that the repository owner can set a default value for this setting effecting the *next* Release file, not the current one, which provides a functionality similar "HTTP Public Key Pinning". The pinning is in effect as long as the (then old) Release file is considered valid, but it is also ignored if the Release file has no Valid-Until at all.
2016-05-01support multiple fingerprints in signed-byDavid Kalnischkies
A keyring file can include multiple keys, so its only fair for transitions and such to support multiple fingerprints as well.
2016-05-01gpgv: cleanup statusfd parsing a bitDavid Kalnischkies
We parse the messages we receive into two big categories: Most of the messages have a keyid as well as a userid and as they are errors we want to show the userids as well. The other category is also errors, but have no userid (like NO_PUBKEY). Explicitly expressing this in code should make it a bit easier to look at and it also help in dropping additional fields or just the newline at the end consistently. Git-Dch: Ignore
2016-05-01don't show NO_PUBKEY warning if repo is signed by another keyDavid Kalnischkies
Daniel Kahn Gillmor highlights in the bugreport that security isn't improving by having the user import additional keys – especially as importing keys securely is hard. The bugreport was initially about dropping the warning to a notice, but in given the previously mentioned observation and the fact that we weren't printing a warning (or a notice) for expired or revoked keys providing a signature we drop it completely as the code to display a message if this was the only key is in another path – and is considered critical. Closes: 618445
2016-05-01gpgv: handle expired sig as worthlessDavid Kalnischkies
Signatures on data can have an expiration date, too, which we hadn't handled previously explicitly (no problem – gpg still has a non-zero exit code so apt notices the invalid signature) so the error message wasn't as helpful as it could be (aka mentioning the key signing it).
2016-05-01gpgv: use EXPKEYSIG instead of KEYEXPIREDDavid Kalnischkies
The upstream documentation says about KEYEXPIRED: "This status line is not very useful". Indeed, it doesn't mention which key is expired, and suggests to use the other message which does.
2016-05-01show StateCache flags in Pkg debug prettyprintDavid Kalnischkies
This basically introduces ~33 flags in the output, but a package can have only ~11 of them displayed at the same time. There is quiet a bit of duplication also (an uninstalled package is by definition a newinstall if its getting installed), but as this is debug output we are better of showing them all in case one of them isn't set in a way it is supposed to be set. Git-Dch: Ignore
2016-04-30zh_TW.po: remove several fuzzy tags after reviewZhou Mo
2016-04-28deb822: Restore support for <multivalue>-{Add,Remove}James McCoy
Redesign of multivalue options in 463c8d801595ce5ac94d7c032264820be7434232 caused the parser to look for <multivalue>{Add,Remove} (no hyphen) instead of the expected <multivalue>-{Add,Remove}.
2016-04-28factor out Pkg/DepIterator prettyprinters into own headerDavid Kalnischkies
The old prettyprinters have only access to the struct they pretty print, which isn't enough usually as we want to know for a package also a bit of state information like which version is the candidate. We therefore need to pull the DepCache into context and hence use a temporary struct which is printed instead of the iterator itself.
2016-04-28deprecate confusing Pkg.CandVersion() methodDavid Kalnischkies
This method does not return the 'current' candidate of the DepCache which would be most expected, but instead returns the version which would be candidate in a default-only policy setting – aka ignoring apt_preferences settings and co.
2016-04-28respect user pinning in M-A:same version (un)screwingDavid Kalnischkies
Using Pkg.CandVersion() here is wrong as its implementation will return a candidate based just on the default policy settings ignoring user preferences and otherwise set candidates (aka: it sidesteps the pkgDepCache). This causes M-A:same libraries to be detected as screwed even through they aren't, so that they end up being kept back. Reported-By: Felipe Sateler on IRC
2016-04-28FileFd: avoid further writing if file failedDavid Kalnischkies
If the file is in a failed state there is no point in trying to flush out the buffers as the file is to be discarded anyhow & its likely all this flushing is producing is additional error messages. Git-Dch: Ignore
2016-04-27Merge branch 'fix-https-noproxy' of github.com:patcable/aptJulian Andres Klode
2016-04-27refactored no_proxy code to work regardless of where https proxy is setPatrick Cable
when using the https transport mechanism, $no_proxy is ignored if apt is getting it's proxy information from $https_proxy (as opposed to Acquire::https::Proxy somewhere in apt config). if the source of proxy information is Acquire::https::Proxy set in apt.conf (or apt.conf.d), then $no_proxy is honored.
2016-04-25private-show: Get rid of old policy support codeJulian Andres Klode
This does not make much sense anymore, now that we dropped the old candidate ver algorithm.
2016-04-25policy: Remove TODO for replacing old GetCandidateVer()Julian Andres Klode
Gbp-Dch: ignore
2016-04-25policy: Get rid of old (pre-1.1) GetCandidateVer algorithmJulian Andres Klode
Bye bye old friend. You're in one Ubuntu LTS release for compat testing, now we do not need you anymore.
2016-04-25restore pinning to min/max value of shortDavid Kalnischkies
Broken in the previous commit (69cea1ef2cfda3c4da79fd756a8edaf2be26998e). Adding a test and a comment to avoid future embarrassment. Git-Dch: Ignore Reported-By: Julian Andres Klode on IRC
2016-04-25give rc-status packages a pin of -1David Kalnischkies
It would previously return a pin of 0, which is an invalid value, but the intend is that versions which are only in the dpkg/status file can't be selected for installation (= can't be a candidate) which is what a negative pin assures. This helps with the communication to EDSP solvers as they neither know about the rc-state (yet) nor that they shouldn't choose this version. Ideally they shouldn't be told about such versions at all as there is nothing to be solved here, but we will get there eventually.
2016-04-25edsp: ask policy engine for the pin of the version directlyDavid Kalnischkies
2016-04-25use the same redirection mirror for all index filesDavid Kalnischkies
Redirection services like httpredir.debian.org tend to use a set of mirrors from which they pick a mirror at "random" for each requested file, which is usually benefitial for the download of debs, but for the index files this can quickly cause problems (aka hashsum mismatches) if the two (or more) mirrors involved are only slightly out-of-sync. This commit "resolves" this issue by using the mirror we ended up using to get the (signed) Release file directly to get the index files belonging to this Release file instead of asking the redirection service which eliminates the risk of hitting out-of-sync mirrors. As an obvious downside the redirection service can't serve partial mirrors anymore for indexes and the download of indexes indexed in the same Release file can't be done in parallel (from different mirrors). This does not effect the download of non-index files like deb-files as out-of-sync mirrors aren't a huge problem there, so the parallel download outweights a potentially 404 error (also because this causes no errenous downloads while hashsum mismatches download the entire file before finding out that it was pointless). The rational for this is that indexes are relative to the Release file. If we would be talking about a HTML page including images, such a behaviour is obvious and intended – not doing it means in the best case a bunch of "useless" requests which will all be answered with a redirect.
2016-04-25show more details for "Writing more data" errors, tooDavid Kalnischkies
They are the small brothers of the hashsum mismatch, so they deserve a similar treatment even through we have for architectual reasons not a much to display as for hashsum mismatches for now.
2016-04-25show more details for "Hash Sum mismatch" errorsDavid Kalnischkies
Users tend to report these errors with just this error message… not very actionable and hard to figure out if this is a temporary or 'permanent' mirror-sync issue or even the occasional apt bug. Showing the involved hashsums and modification times should help in triaging these kind of bugs – and eventually we will have less of them via by-hash. The subheaders aren't marked for translation for now as they are technical glibberish and probably easier to deal with if not translated. After all, our iconic "Hash Sum mismatch" is translated at least. These additions were proposed in #817240 by Peter Palfrader.
2016-04-25format multiline errors properly in acquire progressDavid Kalnischkies
Together with the GlobalError change this allows us to add errors spanning multiple lines, just that we control GlobalError while the acquire progress is dealt with potentially by individual clients which might or might not need to be adapted. This isn't critical through as it either just works as expected anyhow or is a minor styling thing (after all, all this commit does it add two spaces to indent the lines a bit…).
2016-04-25drop empty line from fetch errorDavid Kalnischkies
This is a duplicate of sorts of 0efb29eb36184bbe6de7b1013d1898796d94b171 which is the a lot more frequent case of this error – and also a duplicate of this error message, just without the \n at the end. Git-Dch: Ignore
2016-04-25properly format multiline error messagesDavid Kalnischkies
2016-04-25sanify unused ReportMirrorFailure a tiny bitDavid Kalnischkies
Calling the (non-existent) reporter multiple times for the same error with different codes for the same error (e.g. hashsum) is a bit strange. It also doesn't need to be a public API. Ideally that would all look and behave slightly different, but we will worry about that at the time this is actually (planed to be) used somewhere… Git-Dch: Ignore
2016-04-25don't ask server if we have entire file in partial/David Kalnischkies
We have this situation in cases were parts of the transaction are refused (e.g. in a hashsum mismatch) and rerun the update (e.g. in the hope that we get a mirror which is synced this time). Previously we would ask the server with an if-range and in the best case recieve a 416 in response (less featureful server might end up giving us the entire file again or we get the wrong file this time giving us a hashsum mismatch…), which is a waste of time if we know already by checking the hashsums that we got the complete and correct file.
2016-04-25add dep11 files to default Release patternsDavid Kalnischkies
2016-04-25make random acquire queues work less randomDavid Kalnischkies
Queues feeding workers like rred are created in a random pattern to get a few of them to run in parallel – but if we already have an idling queue we don't need to assign it to a (potentially new) random queue as that saves us the (agruably small) overhead of starting up a new queue, avoids adding jobs to an already busy queue while others idle and as a bonus reduces the size of debug logs a bit. We also keep starting new queues now until we reach our limit before we assign work at random to them, which should give us a more effective utilisation overall compared to potentially adding work to busy queues while we haven't reached our queue limit yet.