summaryrefslogtreecommitdiff
path: root/apt-pkg/deb
AgeCommit message (Collapse)Author
2016-06-29write auto-bits before calling dpkg & again after if neededDavid Kalnischkies
Writing first means that even in the event of a power-failure the autobit is saved for future processing instead of "forgotten" so that the package is treated as manually installed. In some cases we have to re-run the writing after dpkg is done through as dpkg can let packages disappear and in such cases apt will move autobits around (or in that case non-autobits) which we need to store.
2016-06-29Revert "travis: use gcc-5 instead of gcc(-4.8)"David Kalnischkies
This reverts commit 2b8221d66a8284042fc53c7bbb14bb9750e9137f. Avoiding the use of GCC >= 5 stuff lets use go back to 4.8 simplifying the travis setup again as well as reducing the backport requirements in general. This is possible because the std::get_time use requiring GCC >= 5 in 9febc2b238e1e322dce1f94ecbed46d595893b52 was replaced by handrolling it in 1d742e01470bba27715a8191c50adde4b39c2f19, so the remaining uses are just small conviniences we can do without. Gbp-Dch: Ignore
2016-06-28Fix buffer overflow in debListParser::VersionHash()Julian Andres Klode
If a package file is formatted in a way that that no space follows a deprecated "<", we would reformat it to "<=" and increase the length of the output by 1, which can break. Under normal circumstances with "<=" this should not be an issue. Closes: #828812
2016-06-22add insecure (and weak) allow-options for sources.listDavid Kalnischkies
Weak had no dedicated option before and Insecure and Downgrade were both global options, which given the effect they all have on security is rather bad. Setting them for individual repositories only isn't great but at least slightly better and also more consistent with other settings for repositories.
2016-06-22ensure filesize of deb is included in the hashes listDavid Kalnischkies
Filesize is a silly hash all by itself, but in combination with others it can be a strong opponent, so ensuring that it is in the list of hashes and hence checked by the normal course of action the acquire process takes is a good thing.
2016-06-22handle weak-security repositories as unauthenticatedDavid Kalnischkies
APT can be forced to deal with repositories which have no security features whatsoever, so just giving up on repositories which "just" fail our current criteria of good security features is the wrong incentive. Of course, repositories are better of fixing their setup to provide the minimum of security features, but sometimes this isn't possible: Historic repositories for example which do not change (anymore). That also fixes problem with repositories which are marked as trusted, but are providing only weak security features which would fail the parsing of the Release file. Closes: 827364
2016-06-17merge sources.list lines based on Release filenameDavid Kalnischkies
Merging by URI means that having sources lines with different URI methods results in 'strange' warning and error messages, which aren't very friendly from a user point of view as not encoding the method in the filename is effectivly an implementation detail. Merging by filename removes these messages and makes everything "work" even if it isn't working the way it is configured as the indexes aren't acquired over the method given, but over the first method for this release file (which argueably is an implementation detail stemming from the filename encoding, too). So either direction isn't perfectly "right", but personally I prefer "magic" over strange error messages (and doing a full-circle detection of this with its own messages which would need to be translated feels like way too much effort for dubious gain). Closes: 826944
2016-06-14don't use FindFile for external Dir::Bin commandsDavid Kalnischkies
We usually use absolute paths to specific the location of dpkg, apt-key and the like, but there is nothing wrong with using just the command name and instead let exec(3) make the lookup in PATH. We had a wild mixture before, so opting for the more accepting option out of the two seems about right especially as it makes no difference in the default case as apt uses absolute paths.
2016-06-10don't leak dpkg statusfd pipe in debuggingDavid Kalnischkies
Not a big deal to leak fds in debugging mode, but for completeness. Git-Dch: Ignore
2016-06-09remove racy_pselect fallbackDavid Kalnischkies
The comment says it should have been removed with Lenny+1 which is a small while ago already, so it seems like a good time now… And as this is a cleanup commit it also gets right of spurious whitespace at the end of lines, adds missing fold markers and similar busy work. Git-Dch: Ignore
2016-06-08drop Dpkg::MaxArgs in favor of Dpkg::MaxArgsBytesDavid Kalnischkies
We had an old FIXME saying that it is probably pointless to do this if we limit by length of the commandline already and I completely agree. The splitting is bad enough if it must be done, so we should only do it if we have to (as in absolute length of commandline) and, but that is just a remark, it is unlikely that we ever have/had a call triggering this as the default value was ~32000 items…
2016-06-08don't explicitly configure the last round of packagesDavid Kalnischkies
We end our operation by calling "dpkg --configure -a", so instead of running a (big) configure run with all packages mentioned explicitly before this, we simply skip them and let them be handled by this call implicitly. There isn't really an observeable gain to be had here from a speed point, but it helps in avoiding an (uncommon) problem of having a too long commandline passed to dpkg, which we would split up (probably incorrectly).
2016-05-27prevent C++ locale number formatting in text APIsDavid Kalnischkies
Setting the C++ locale via std::locale::global(std::locale("")); which would otherwise default to the default C locale (aka: unaffected by setlocale) effects the formatting of numeric types in IO streams, which for output for humans is perfectly sensible, but breaks our many text interfaces used and parsed by us and others without expecting the numbers to be formatted. Closes: #825396
2016-05-24fix two typos in untranslated errors of libapt-pkgDavid Kalnischkies
Reported-By: lintian: spelling-error-in-binary Git-Dch: Ignore
2016-05-15Normalize Signed-By values by removing trailing commas everywhereJulian Andres Klode
This fixes comparisons where either the stored or the input string have a trailing comma.
2016-05-15Add conflicting Signed-By values to error messageJulian Andres Klode
This hopefully makes debugging things easier.
2016-05-10Strip trailing commas for created signed-by fingerprint listsJulian Andres Klode
This prevented some sources.list entries from working, an example of which can be found in the test.
2016-05-08implement Identifier field for IndexTargetsDavid Kalnischkies
A frontend like apt-file is only interested in a specific set of files and selects those easily via "Created-By". If it supports two locations for those files through it would need to select both and a user would need to know that implementation detail for sources.list configuration. The "Identifier" field is hence introduced which by default has the same value as "Created-By", but can be freely configured – especially it can be used to give two indexes the same identifier.
2016-05-08implement Fallback-Of for IndexTargetsDavid Kalnischkies
Sometimes index files are in different locations in a repository as it is currently the case for Contents files which are per-component in Debian, but aren't in Ubuntu. This has historic reasons and is perhaps changed soon, but such cases of transitions can always happen in the future again, so we should prepare: Introduced is a new field declaring that the current item should only be downloaded if the mentioned item wasn't allowing for transitions without a flagday in clients and archives. This isn't implemented 'simpler' with multiple MetaKeys as items (could) change their descriptions and perhaps also other configuration bits with their location.
2016-05-07download arch:all also for NATIVE_ARCHITECTURE indextargetsDavid Kalnischkies
It looks a bit strange on the outside to have multiple "native architecture", but all is considered an implementation detail and e.g. packages of arch:all are in dependency resolution equal to native packages.
2016-05-07don't construct MetaIndex acquire items with IndexTargetsDavid Kalnischkies
We don't have to initialize the Release files with a set of IndexTargets to acquire, but instead wait for the Release file to be acquired and only then ask which IndexTargets to get. Git-Dch: Ignore
2016-05-03let DPKG_COLORS default to our APT::Color settingDavid Kalnischkies
dpkg can optionally colorize its output since 1.18.5. Currently this defaults to 'never', but it will eventually be 'auto'. It seems reasonable to assume that a user who has enabled/disabled colors in apt will want to have dpkg have the same state regarding color usage. This isn't overriding explicit settings by the user, so in case a user feels strongly about it one way or the other there are options.
2016-05-01support Signed-By in Release files as a sort of HPKPDavid Kalnischkies
Users have the option since apt >= 1.1 to enforce that a Release file is signed with specific key(s) either via keyring filename or fingerprints. This commit adds an entry with the same name and value (except that it doesn't accept filenames for obvious reasons) to the Release file so that the repository owner can set a default value for this setting effecting the *next* Release file, not the current one, which provides a functionality similar "HTTP Public Key Pinning". The pinning is in effect as long as the (then old) Release file is considered valid, but it is also ignored if the Release file has no Valid-Until at all.
2016-05-01support multiple fingerprints in signed-byDavid Kalnischkies
A keyring file can include multiple keys, so its only fair for transitions and such to support multiple fingerprints as well.
2016-04-03don't leak on error in listparser creationDavid Kalnischkies
Git-Dch: Ignore Reported-By: gcc -fsanitize=address
2016-03-25drop confusing comma from no strong hash messageDavid Kalnischkies
2016-03-14enforce verify of filesize in 'apt-get source'David Kalnischkies
The structure we parse the data into has a dedicated size field, but it tends to be easier to handle it as a (very weak) checksum.
2016-03-14streamline dpkgpm cleanup-handlingDavid Kalnischkies
The (unlikely) waitpid failure case should fallthrough the code just like the other failures (and successes) instead of taking a shortcut avoiding all the cleanup (progress) and finishing touches (log, state). This also delays the cleanup of the progress until apt is really done with everything and "just" has the post-invokes left to do, so the period of 'apt looks finished as it stopped the progress' and 'apt really finished as I have the shell-prompt back' is shorter even if there is no progress reported anymore, so the bar lingers at 100%… Ideally even the post-invokes would be covered by progress, but they can have their own output and dealing with that could be hard. Git-Dch: Ignore
2016-03-07Fix several typosVeres Lajos
This effectively merges branch 'typofixes-vlajos-20150807' of github.com:vlajos/apt with the following commit: commit 13cacb3e2e2352ba701e769fc889e3344fabbf7e Author: Veres Lajos <vlajos@gmail.com> Date: Sun Aug 9 00:12:53 2015 +0100 typofix - https://github.com/vlajos/misspell_fixer It has been rebased for a better commit message.
2016-03-06get group again after potential remap in Source: parseDavid Kalnischkies
Mysteriously segfaults only on i386 for me, but at least one reporter had the same behavior and it makes sense that this is the problem as the parsing of Source: was fixed in 1.2.2 – before the not remapped group was not used. We don't use our usual Dynamic<> trick here as we don't have it in the parser. Its a bit of a layer violation to do this parsing here, but its how it is always was… Until next time with this lovely kind of problem. Closes: 812251 Thanks: Francesco Poli and Marc Haber for testdata.
2016-02-26Add missing numeric includes in files using std::accumulate()Julian Andres Klode
Reported-By: Helmut Grohne on IRC
2016-01-27only warn about missing/invalid Date field for nowDavid Kalnischkies
The Date field in the Release file is useful to avoid allowing an attacker to 'downgrade' a user to earlier Release files (and hence to older states of the archieve with open security bugs). It is also needed to allow a user to define min/max values for the validation of a Release file (with or without the Release file providing a Valid-Until field). APT wasn't formally requiring this field before through and (agrueable not binding and still incomplete) online documentation declares it optional (until now), so we downgrade the error to a warning for now to give repository creators a bit more time to adapt – the bigger ones should have a Date field for years already, so the effected group should be small in any case. It should be noted that earlier apt versions had this as an error already, but only showed it if a Valid-Until field was present (or the user tried to used the configuration items for min/max valid-until). Closes: 809329
2016-01-26convert Version() and Architecture() to APT::StringViewDavid Kalnischkies
Part of hidden classes, so conversion is abi-free. Git-Dch: Ignore
2016-01-26remove unused Description methods in listparsersDavid Kalnischkies
These virtual methods are implemented in hidden classes, so we can drop them without breaking the ABI. Git-Dch: Ignore
2016-01-26parse version correctly from binary Source fieldDavid Kalnischkies
In commit a221efc331693f8905da870141756c892911c433 I promoted the source package name and version to the binary cache for faster access by e.g. EDSP, but due to changing the interpretation length to soon we always ignored the version part of the Source field, so that packages ended up having the binary version as source version – which while usually just fine it is wrong for binary rebuilds. Closes: 812492
2016-01-25reimplement build-dep via apts normal resolverDavid Kalnischkies
build-dep was implemented by parsing the build-dependencies of a package and figuring out which packages to install/remove based on this. That means that for the first level of dependencies build-dep was implementing its very own resolver with all the benefits (aka: bugs) this gives us for not using the existing resolver for all levels. Making this work involves generating a dummy binary package with fitting Depends and Conflicts and as we can't create them out of thin air the cache generation needs to be involved so we end up writing a Packages file which we want to parse – after we have parsed the other Packages files already. With .dsc/.deb files we could add them before we started parsing anything. With a bit of care we can avoid generating too much data we have to throw away again (as many parts assume that e.g. the count of packages doesn't change midair), so that on a speed front there shouldn't be much of a difference, but output can be slightly confusing as if we have a completely valid cache on disk the "Reading package lists... Done" is printed two times – but apt is pretty quick about it in that case. Closes: #137560, #444930, #489911, #583914, #728317, #812173
2016-01-25treat an empty dependency field just like it doesn't existDavid Kalnischkies
Git-Dch: Ignore
2016-01-25Use c++11 for loop in AptHistoryRequestingUserMichael Vogt
Git-Dch: ignore
2016-01-25Store "Requested-By" user in history.log in a simpler formatMichael Vogt
Git-Dch: ignore Thanks: David Kalnischkies
2016-01-25Log calling SUDO_USER or PKEXEC_UID in history.logMichael Vogt
Thanks: Thomas Reusch
2016-01-15use APT::StringView for GrabWordDavid Kalnischkies
Git-Dch: Ignore
2016-01-14fix M-A:foreign provides creation for unknown archsDavid Kalnischkies
Architectures for packages which do not belong to the native nor a foreign architecture (dubbed barbarian for now) which are marked M-A:foreign still provide in their own architecture even if not for others. Also, other M-A:foreign (and allowed) packages provide in these barbarian architectures.
2016-01-08debListParser: Convert another ParseDepends to StringViewJulian Andres Klode
I overlooked this Gbp-Dch: ignore
2016-01-08keep compressed indexes in a low-cost formatDavid Kalnischkies
Downloading and storing are two different operations were different compression types can be preferred. For downloading we provide the choice via Acquire::CompressionTypes::Order as there is a choice to be made between download size and speed – and limited by whats available in the repository. Storage on the other hand has all compressions currently supported by apt available and to reduce runtime of tools accessing these files the compression type should be a low-cost format in terms of decompression. apt traditionally stores its indexes uncompressed on disk, but has options to keep them compressed. Now that apt downloads additional files we also deal with files which simply can't be stored uncompressed as they are just too big (like Contents for apt-file). Traditionally they are downloaded in a low-cost format (gz) as repositories do not provide other formats, but there might be even lower-cost formats and for download we could introduce higher-cost in the repositories. Downloading an entire index potentially requires recompression to another format, so an update takes potentially longer – but big files are usually updated via pdiffs which has to de- and re-compress anyhow and does it on the fly anyhow, so there is no extra time needed and in general it seems to be benefitial to invest the time in update to save time later on file access.
2016-01-08AvailableDescriptionLanguages: Use one string for all iterationsJulian Andres Klode
Do not create strings within the loop, that creates one string per language and does more work than needed. Instead, reserve enough space at the beginning and assign the prefix, and then resize and append inside the loop. Also call exists with the string itself instead of the c_str(), this means that the lookup uses the size information in the string now and does not have to call strlen() on it.
2016-01-08Replace compare() == 0 checks with this == other checksJulian Andres Klode
This improves performance, as we now can ignore unequal strings based on their length already. Gbp-Dch: ignore
2016-01-07Switch performance critical code to use APT::StringViewJulian Andres Klode
This improves performance of the cache generation on my ARM platform (4x Cortex A15) by about 10% to 20% from 2.35-2.50 to 2.1 seconds.
2016-01-02support comments in debian/control parsingDavid Kalnischkies
Now (55153bf94ff28a23318e79aa48242244c4d82b3c) that pkgTagFile can be told to deal with all sorts of comments we can use this mode to parse dsc (as by catch) and debian/control files properly even in the wake of multiline fields spliced with comments like Build-Depends. Closes: 806775
2015-12-27allow repositories to forbid arch:all for specific index targetsDavid Kalnischkies
Debian has a Packages file for arch:all already, but the arch:any files contain arch:all packages as well, so downloading it would be a total waste of resources. Getting this solved is on the list of things to do, but it is also the hardest part – for index targets like Contents the situation is much easier and less server/client implementations are involved so we might not want to stall them. A repository can now declare via: No-Support-for-Architecture-all: Packages that even if an arch:all Packages exists, it shouldn't be downloaded, so that support for Contents files can be added now. See also 1dd20368486820efb6ef4476ad739e967174bec4 for the implementation of downloading arch:all index targets, which this is limiting. The field uses the name of the target from the apt configuration for simplicity and is negative by design as this field is intended to be supported/needed only for a "short" time (one or two Debian releases). While this commit theoretically supports any target, its expected to only see "Packages" as a value in reality.
2015-12-27ParseDepends: Mark branches for build-dep parsing as unlikelyJulian Andres Klode
We do not see those branches at all during normal mode of operation (that is, during cache generation), so tell the compiler about it.