summaryrefslogtreecommitdiff
path: root/apt-pkg/deb
AgeCommit message (Collapse)Author
2019-03-01Add a Packages-Require-Authorization Release file fieldJulian Andres Klode
This new field allows a repository to declare that access to packages requires authorization. The current implementation will set the pin to -32768 if no authorization has been provided in the auth.conf(.d) files. This implementation is suboptimal in two aspects: (1) A repository should behave more like NotSource repositories (2) We only have the host name for the repository, we cannot use paths yet. - We can fix those after an ABI break. The code also adds a check to acquire-item.cc to not use the specified repository as a download source, mimicking NotSource. (cherry picked from commit c2b9b0489538fed4770515bd8853a960b13a2618) LP: #1814727 (cherry picked from commit d75162bc67d5a1a690eb2a8747d31ad68353823e) (cherry picked from commit 19075f52174199fe7665334ad1815c747c26c10b)
2018-10-09Set DPKG_FRONTEND_LOCKED when running {pre,post}-invoke scriptsJulian Andres Klode
Some post-invoke scripts install packages, which fails because the environment variable is not set. This sets the variable for all three kinds of scripts {pre,post-}invoke and pre-install-pkgs, but we will only allow post-invoke at a later time. Gbp-Dch: full (cherry picked from commit 6675601c81de85b40dc89772c1d6d17f1811c5ba) LP: #1796808 (cherry picked from commit 0ce6bc5ec70ff84eb2829f91f286a93687e77a46)
2018-09-28Set DPKG_FRONTEND_LOCKED as needed when doing selection changesJulian Andres Klode
We forgot to set the variable for the selection changes. Let's set it for that and some other dpkg calls. Regression-Of: c2c8b4787b0882234ba2772ec7513afbf97b563a (cherry picked from commit 55489885b51b02b7f74e601a393ecaefd1f71f9c) (cherry picked from commit d66bd6e5e9ae96676e805cce43937a0528cebe1b)
2018-09-28Add support for dpkg frontend lockJulian Andres Klode
The dpkg frontend lock is a lock dpkg tries to acquire except if the frontend already acquires it. This fixes a race condition in the install command where the dpkg lock is not held for a short period of time between different dpkg invocations. For this reason we also define an environment variable DPKG_FRONTEND_LOCKED for dpkg invocations so dpkg knows not to try to acquire the frontend lock because it's held by a parent process. We can set DPKG_FRONTEND_LOCKED only if the frontend lock really is held; that is, if our lock count is greater than 0 - otherwise an apt client not using the LockInner family of functions would run dpkg without the frontend lock set, but with DPKG_FRONTEND_LOCKED set. Such a process has a weaker guarantee: Because dpkg would not lock the frontend lock either, the process is prone to the existing races, and, more importantly, so is a new style process. Closes: #869546 [fixups: fix error messages, add public IsLocked() method, and make {Un,}LockInner return an error on !debSystem] (cherry picked from commit c2c8b4787b0882234ba2772ec7513afbf97b563a) LP: #1781169 (cherry picked from commit 6c0c94ed32b8e679b14b0f89b51c1c336dc0ab9c)
2018-06-25Fix lock counting in debSystemJulian Andres Klode
debSystem uses a reference counted lock, so you can acquire it multiple times in your applications, possibly nested. Nesting locks causes a fd leak, though, as we only increment the lock count when we already have locked twice, rather than once, and hence when we call lock the second time, instead of increasing the lock count, we open another lock fd. This fixes the code to check if we have locked at all (> 0). There is no practical problem here aside from the fd leak, as closing the new fd releases the lock on the old one due to the weird semantics of fcntl locks. (cherry picked from commit 79f012bd09ae99d4c9d63dc0ac960376b5338b32) (cherry picked from commit 1edcb718293f24ad190703a345f8f868b6e3bcc4) LP: #1778547
2017-06-19Fix parsing of or groups in build-deps with ignored packagesJulian Andres Klode
If the last alternative(s) of an Or group is ignored, because it does not match an architecture list, we would end up keeping the or flag, effectively making the next AND an OR. For example, when parsing (on amd64): debhelper (>= 9), libnacl-dev [amd64] | libnacl-dev [i386] => debhelper (>= 9), libnacl-dev | Which can cause python-apt to crash. Even worse: debhelper (>= 9), libnacl-dev [amd64] | libnacl-dev [i386], foobar => debhelper (>= 9), libnacl-dev [amd64] | foobar By setting the previous alternatives Or flag to the current Or flag if the current alternative is ignored, we solve the issue. LP: #1694697 (cherry picked from commit 7ddf958e370d13f93edc6923bee289b2f6444b41) (cherry picked from commit 423ba4a958b9da02926e586bf59995817cafc32a)
2017-02-22Do not package names representing .dsc/.deb/... filesJulian Andres Klode
In the case of build-dep and other commands where a file can be passed we must make sure not to normalize the path name as that can have odd side effects, or well, cause the operation to do nothing. Test for build-dep-file is adjusted to perform the vcard check once as "vcard" and once as "VCard", thus testing that this solves the reported bug. We inline the std::transform() and optimize it a bit to not write anything in the common case (package names are defined to be lowercase, the whole transformation is just for names that should not exist...) to counter the performance hit of the added find() call (it's about 0.15% more instructions than with the existing transform, but we save about 0.67% in writes...). Closes: #854794 (cherry picked from commit 85ee4036c68d8ecd2c973d413a17aca81380900b) (cherry picked from commit 83e6e1a8fc942668f9a01906cb8349fb70a45b3d)
2017-02-22avoid producing invalid options if repo has no hostDavid Kalnischkies
This can happen e.g. for file: repositories. There is no inherent problem with setting such values internally, but its bad style, forbidden in the manpage and could be annoying in the future. Gbp-Dch: Ignore (cherry picked from commit 44ecb8c3579e5ae8828f83530e4151a0ff84d5d6) (cherry picked from commit fec19de5e786564ed8699b38310f7d1a7c348c01)
2016-10-05VersionHash: Do not skip too long dependency linesJulian Andres Klode
If the dependency line does not contain spaces in the repository but does in the dpkg status file (because dpkg normalized the dependency list), the dpkg line might be longer than the line in the repository. If it now happens to be longer than 1024 characters, it would be skipped, causing the hashes to be out of date. Note that we have to bump the minor cache version again as this changes the format slightly, and we might get mismatches with an older src cache otherwise. Fixes Debian/apt#23 (cherry picked from commit 708e2f1fe99e6f067292bc909f03f12c181e4798)
2016-10-05Fix segfault and out-of-bounds read in Binary fieldsJulian Andres Klode
If a Binary field contains one or more spaces before a comma, the code produced a segmentation fault, as it accidentally set a pointer to 0 instead of the value of the pointer. If the comma is at the beginning of the field, the code would create a binStartNext that points one element before the start of the string, which is undefined behavior. We also need to check that we do not exit the string during the replacement of spaces before commas: A string of the form " ," would normally exit the boundary of the Buffer: binStartNext = offset 1 ',' binEnd = offset 0 ' ' isspace_ascii(*binEnd) = true => --binEnd => binEnd = - 1 We get rid of the problem by only allowing spaces to be eliminated if they are not the first character of the buffer: binStartNext = offset 1 ',' binEnd = offset 0 ' ' binEnd > buffer = false, isspace_ascii(*binEnd) = true => exit loop => binEnd remains 0 (cherry picked from commit ce6cd75dc367b92f65e4fb539dd166d0f3361f8c)
2016-10-05Ignore SIGINT and SIGQUIT for Pre-Install hooksJulian Andres Klode
Instead of erroring out when receiving a SIGINT, let the child deal with it - we'll error out anyway if the child exits with an error or due to the signal. Also ignore SIGQUIT, as system() ignores it. This basically fixes Bug #832593, but: we are running the hooks via sh -c. Some shells exit with a signal error even if the command they are executing catches the signal and exits successfully. So far, this has been noticed on dash, which unfortunately, is our default shell. Example: $ cat trap.sh trap 'echo int' INT; sleep 10; exit 0 $ if dash -c ./trap.sh; then echo OK: $?; else echo FAIL: $?; fi ^Cint FAIL: 130 $ if mksh -c ./trap.sh; then echo OK: $?; else echo FAIL: $?; fi ^Cint OK: 0 $ if bash -c ./trap.sh; then echo OK: $?; else echo FAIL: $?; fi ^Cint OK: 0 (cherry picked from commit a6ae3d3df490e7a5a1c8324ba9dc2e63972b1529)
2016-10-05pass --force-remove-essential to dpkg only if neededDavid Kalnischkies
APT (usually) knows which package is essential or not, so we can avoid passing this force flag to dpkg unconditionally if the user hasn't chosen a non-default essential handling obscuring the information. (cherry picked from commit d3930f8716f439c229cd3d11813823d847a2ecff)
2016-08-31reinstalling local deb file is no downgradeDavid Kalnischkies
If we have a (e.g. locally built) deb file installed and do try to install it again apt complained about this being a downgrade, but it wasn't as it is the very same version… it was just confused into not merging the versions together which looks like a downgrade then. The same size assumption is usually good, but given that volatile files are parsed last (even after the status file) the base assumption no longer holds, but is easy to adept without actually changing anything in practice. (cherry picked from commit e7edb2fef8370d54a4b8e5a01266e6eda81ef84e)
2016-08-31write auto-bits before calling dpkg & again after if neededDavid Kalnischkies
Writing first means that even in the event of a power-failure the autobit is saved for future processing instead of "forgotten" so that the package is treated as manually installed. In some cases we have to re-run the writing after dpkg is done through as dpkg can let packages disappear and in such cases apt will move autobits around (or in that case non-autobits) which we need to store. (cherry picked from commit 309f497b7280a45e3626493318adb6d39ba5c69b)
2016-08-31Fix buffer overflow in debListParser::VersionHash()Julian Andres Klode
If a package file is formatted in a way that that no space follows a deprecated "<", we would reformat it to "<=" and increase the length of the output by 1, which can break. Under normal circumstances with "<=" this should not be an issue. Closes: #828812 (cherry picked from commit b6e9756ca03ec887ef1d0bc8e38f63c29db7a365)
2016-06-22ensure filesize of deb is included in the hashes listDavid Kalnischkies
Filesize is a silly hash all by itself, but in combination with others it can be a strong opponent, so ensuring that it is in the list of hashes and hence checked by the normal course of action the acquire process takes is a good thing. (cherry picked from commit 5da51e0e2da3f055306562d38103b06a23d81719)
2016-06-01prevent C++ locale number formatting in text APIsDavid Kalnischkies
Setting the C++ locale via std::locale::global(std::locale("")); which would otherwise default to the default C locale (aka: unaffected by setlocale) effects the formatting of numeric types in IO streams, which for output for humans is perfectly sensible, but breaks our many text interfaces used and parsed by us and others without expecting the numbers to be formatted. Closes: #825396 (cherry picked from commit b58e2c7c56b1416a343e81f9f80cb1f02c128e25)
2016-04-03don't leak on error in listparser creationDavid Kalnischkies
Git-Dch: Ignore Reported-By: gcc -fsanitize=address
2016-03-25drop confusing comma from no strong hash messageDavid Kalnischkies
2016-03-14enforce verify of filesize in 'apt-get source'David Kalnischkies
The structure we parse the data into has a dedicated size field, but it tends to be easier to handle it as a (very weak) checksum.
2016-03-14streamline dpkgpm cleanup-handlingDavid Kalnischkies
The (unlikely) waitpid failure case should fallthrough the code just like the other failures (and successes) instead of taking a shortcut avoiding all the cleanup (progress) and finishing touches (log, state). This also delays the cleanup of the progress until apt is really done with everything and "just" has the post-invokes left to do, so the period of 'apt looks finished as it stopped the progress' and 'apt really finished as I have the shell-prompt back' is shorter even if there is no progress reported anymore, so the bar lingers at 100%… Ideally even the post-invokes would be covered by progress, but they can have their own output and dealing with that could be hard. Git-Dch: Ignore
2016-03-07Fix several typosVeres Lajos
This effectively merges branch 'typofixes-vlajos-20150807' of github.com:vlajos/apt with the following commit: commit 13cacb3e2e2352ba701e769fc889e3344fabbf7e Author: Veres Lajos <vlajos@gmail.com> Date: Sun Aug 9 00:12:53 2015 +0100 typofix - https://github.com/vlajos/misspell_fixer It has been rebased for a better commit message.
2016-03-06get group again after potential remap in Source: parseDavid Kalnischkies
Mysteriously segfaults only on i386 for me, but at least one reporter had the same behavior and it makes sense that this is the problem as the parsing of Source: was fixed in 1.2.2 – before the not remapped group was not used. We don't use our usual Dynamic<> trick here as we don't have it in the parser. Its a bit of a layer violation to do this parsing here, but its how it is always was… Until next time with this lovely kind of problem. Closes: 812251 Thanks: Francesco Poli and Marc Haber for testdata.
2016-02-26Add missing numeric includes in files using std::accumulate()Julian Andres Klode
Reported-By: Helmut Grohne on IRC
2016-01-27only warn about missing/invalid Date field for nowDavid Kalnischkies
The Date field in the Release file is useful to avoid allowing an attacker to 'downgrade' a user to earlier Release files (and hence to older states of the archieve with open security bugs). It is also needed to allow a user to define min/max values for the validation of a Release file (with or without the Release file providing a Valid-Until field). APT wasn't formally requiring this field before through and (agrueable not binding and still incomplete) online documentation declares it optional (until now), so we downgrade the error to a warning for now to give repository creators a bit more time to adapt – the bigger ones should have a Date field for years already, so the effected group should be small in any case. It should be noted that earlier apt versions had this as an error already, but only showed it if a Valid-Until field was present (or the user tried to used the configuration items for min/max valid-until). Closes: 809329
2016-01-26convert Version() and Architecture() to APT::StringViewDavid Kalnischkies
Part of hidden classes, so conversion is abi-free. Git-Dch: Ignore
2016-01-26remove unused Description methods in listparsersDavid Kalnischkies
These virtual methods are implemented in hidden classes, so we can drop them without breaking the ABI. Git-Dch: Ignore
2016-01-26parse version correctly from binary Source fieldDavid Kalnischkies
In commit a221efc331693f8905da870141756c892911c433 I promoted the source package name and version to the binary cache for faster access by e.g. EDSP, but due to changing the interpretation length to soon we always ignored the version part of the Source field, so that packages ended up having the binary version as source version – which while usually just fine it is wrong for binary rebuilds. Closes: 812492
2016-01-25reimplement build-dep via apts normal resolverDavid Kalnischkies
build-dep was implemented by parsing the build-dependencies of a package and figuring out which packages to install/remove based on this. That means that for the first level of dependencies build-dep was implementing its very own resolver with all the benefits (aka: bugs) this gives us for not using the existing resolver for all levels. Making this work involves generating a dummy binary package with fitting Depends and Conflicts and as we can't create them out of thin air the cache generation needs to be involved so we end up writing a Packages file which we want to parse – after we have parsed the other Packages files already. With .dsc/.deb files we could add them before we started parsing anything. With a bit of care we can avoid generating too much data we have to throw away again (as many parts assume that e.g. the count of packages doesn't change midair), so that on a speed front there shouldn't be much of a difference, but output can be slightly confusing as if we have a completely valid cache on disk the "Reading package lists... Done" is printed two times – but apt is pretty quick about it in that case. Closes: #137560, #444930, #489911, #583914, #728317, #812173
2016-01-25treat an empty dependency field just like it doesn't existDavid Kalnischkies
Git-Dch: Ignore
2016-01-25Use c++11 for loop in AptHistoryRequestingUserMichael Vogt
Git-Dch: ignore
2016-01-25Store "Requested-By" user in history.log in a simpler formatMichael Vogt
Git-Dch: ignore Thanks: David Kalnischkies
2016-01-25Log calling SUDO_USER or PKEXEC_UID in history.logMichael Vogt
Thanks: Thomas Reusch
2016-01-15use APT::StringView for GrabWordDavid Kalnischkies
Git-Dch: Ignore
2016-01-14fix M-A:foreign provides creation for unknown archsDavid Kalnischkies
Architectures for packages which do not belong to the native nor a foreign architecture (dubbed barbarian for now) which are marked M-A:foreign still provide in their own architecture even if not for others. Also, other M-A:foreign (and allowed) packages provide in these barbarian architectures.
2016-01-08debListParser: Convert another ParseDepends to StringViewJulian Andres Klode
I overlooked this Gbp-Dch: ignore
2016-01-08keep compressed indexes in a low-cost formatDavid Kalnischkies
Downloading and storing are two different operations were different compression types can be preferred. For downloading we provide the choice via Acquire::CompressionTypes::Order as there is a choice to be made between download size and speed – and limited by whats available in the repository. Storage on the other hand has all compressions currently supported by apt available and to reduce runtime of tools accessing these files the compression type should be a low-cost format in terms of decompression. apt traditionally stores its indexes uncompressed on disk, but has options to keep them compressed. Now that apt downloads additional files we also deal with files which simply can't be stored uncompressed as they are just too big (like Contents for apt-file). Traditionally they are downloaded in a low-cost format (gz) as repositories do not provide other formats, but there might be even lower-cost formats and for download we could introduce higher-cost in the repositories. Downloading an entire index potentially requires recompression to another format, so an update takes potentially longer – but big files are usually updated via pdiffs which has to de- and re-compress anyhow and does it on the fly anyhow, so there is no extra time needed and in general it seems to be benefitial to invest the time in update to save time later on file access.
2016-01-08AvailableDescriptionLanguages: Use one string for all iterationsJulian Andres Klode
Do not create strings within the loop, that creates one string per language and does more work than needed. Instead, reserve enough space at the beginning and assign the prefix, and then resize and append inside the loop. Also call exists with the string itself instead of the c_str(), this means that the lookup uses the size information in the string now and does not have to call strlen() on it.
2016-01-08Replace compare() == 0 checks with this == other checksJulian Andres Klode
This improves performance, as we now can ignore unequal strings based on their length already. Gbp-Dch: ignore
2016-01-07Switch performance critical code to use APT::StringViewJulian Andres Klode
This improves performance of the cache generation on my ARM platform (4x Cortex A15) by about 10% to 20% from 2.35-2.50 to 2.1 seconds.
2016-01-02support comments in debian/control parsingDavid Kalnischkies
Now (55153bf94ff28a23318e79aa48242244c4d82b3c) that pkgTagFile can be told to deal with all sorts of comments we can use this mode to parse dsc (as by catch) and debian/control files properly even in the wake of multiline fields spliced with comments like Build-Depends. Closes: 806775
2015-12-27allow repositories to forbid arch:all for specific index targetsDavid Kalnischkies
Debian has a Packages file for arch:all already, but the arch:any files contain arch:all packages as well, so downloading it would be a total waste of resources. Getting this solved is on the list of things to do, but it is also the hardest part – for index targets like Contents the situation is much easier and less server/client implementations are involved so we might not want to stall them. A repository can now declare via: No-Support-for-Architecture-all: Packages that even if an arch:all Packages exists, it shouldn't be downloaded, so that support for Contents files can be added now. See also 1dd20368486820efb6ef4476ad739e967174bec4 for the implementation of downloading arch:all index targets, which this is limiting. The field uses the name of the target from the apt configuration for simplicity and is negative by design as this field is intended to be supported/needed only for a "short" time (one or two Debian releases). While this commit theoretically supports any target, its expected to only see "Packages" as a value in reality.
2015-12-27ParseDepends: Mark branches for build-dep parsing as unlikelyJulian Andres Klode
We do not see those branches at all during normal mode of operation (that is, during cache generation), so tell the compiler about it.
2015-12-27debListParser: Do not validate Description-md5 for correctness twiceJulian Andres Klode
The Set() method returns false if the input is no hex number, so simply use that.
2015-12-27debListParser: ParseDepends: Only query native arch if neededJulian Andres Klode
This makes the code parsing architecture lists slower, but on the other hand, improves the more generic case of reading dependencies from Packages files.
2015-12-27Convert most callers of isspace() to isspace_ascii()Julian Andres Klode
This converts all callers that read machine-generated data, callers that might work with user input are not converted.
2015-12-14show a more descriptive error for weak Release filesDavid Kalnischkies
If we can't work with the hashes we parsed from the Release file we display now an error message if the Release file includes only weak hashes instead of downloading the indexes and failing to verify them with "Hash Sum mismatch" even through the hashes didn't mismatch (they were just weak). If for some (unlikely) reason we have got weak hashes only for individual targets we will show a warning to this effect (again, befor downloading and failing the index itself). Closes: 806459
2015-12-11Convert package names from Packages files to lower caseJulian Andres Klode
dpkg does that when reading package files, so we should do the same. This only deals with parsing names from binary package paragraphs, it does not look at source package names and/or the list of binaries in a dsc file. Closes: #807012
2015-12-07Avoid overflow when summing up file sizesJulian Andres Klode
We need to pass 0llu instead of 0 as the init value, otherwise std::accumulate will calculate with ints. Reported-by: Raphaël Hertzog
2015-12-01require explicit paths to dsc/control as we do for deb filesDavid Kalnischkies
Otherwise a user is subject to unexpected content-injection depending on which directory she happens to start apt in. This also cleans up the code requiring less implementation details in build-dep which is always good. Technically, this is an ABI break as we override virtual methods, but that they weren't overridden was a mistake resulting in pure classes, which shouldn't be pure, so they were unusable – and as they are new in 1.1 nobody is using them yet (and hopefully ever as they are borderline implementation details). Closes: 806693