summaryrefslogtreecommitdiff
path: root/apt-pkg/deb
AgeCommit message (Collapse)Author
2018-09-11Reorder progress report messagesDavid Kalnischkies
We are seeing 'processing' messages from dpkg first, so it makes sense to translate them to "Preparing" messages instead of using "Installing" and co to override these shortly after with the "Preparing" messages. The difference isn't all to visible as later messages tend to linger far longer in the display than the ealier ones, but at least in a listing it seems more logical.
2018-09-11Don't expect duplicated dpkg status-fd messagesDavid Kalnischkies
The progress reporting relies on parsing the status reports of dpkg which used to repeat being in the same state multiple times in the same run, but by fixing #365921 it will stop doing so. The problem is in theory just with 'config-files' in case we do purge as this (can) do remove + purge in one step, but we remove this also for the unpack + configure combination althrough we handle these currently in two independent dpkg calls.
2018-09-11Process status-fd completely before finishing dpkg callDavid Kalnischkies
Exiting the processing loop as soon as the dpkg process finishes might leave status-fd lines unprocessed which wasn't much of a problem in the past as the progress would just be slightly off, but now that we us the information also for skipping already done tasks and generate warnings if we didn't see all expected messages we should make sure we seem them all. We still need to exit "early" if dpkg exited unsuccessfully/crashed through as the (remaining) status lines we get could be incomplete.
2018-08-29Don't use invalid iterator in Fallback-Of handlingDavid Kalnischkies
cppcheck reports: (error) Iterator 't' used after element has been erased. The loop is actually fashioned to deal with this (not in the most efficient way, but in simplest and speed isn't really a concern here) IF this codepath had a "break" at the end… so I added one. Note that the tests aren't failing before (and hopefully after) the change as the undefined behavior we encounter is too stable. Thanks: David Binderman for reporting
2018-08-08Set DPKG_FRONTEND_LOCKED as needed when doing selection changesJulian Andres Klode
We forgot to set the variable for the selection changes. Let's set it for that and some other dpkg calls. Regression-Of: c2c8b4787b0882234ba2772ec7513afbf97b563a
2018-08-07Add support for dpkg frontend lockJulian Andres Klode
The dpkg frontend lock is a lock dpkg tries to acquire except if the frontend already acquires it. This fixes a race condition in the install command where the dpkg lock is not held for a short period of time between different dpkg invocations. For this reason we also define an environment variable DPKG_FRONTEND_LOCKED for dpkg invocations so dpkg knows not to try to acquire the frontend lock because it's held by a parent process. We can set DPKG_FRONTEND_LOCKED only if the frontend lock really is held; that is, if our lock count is greater than 0 - otherwise an apt client not using the LockInner family of functions would run dpkg without the frontend lock set, but with DPKG_FRONTEND_LOCKED set. Such a process has a weaker guarantee: Because dpkg would not lock the frontend lock either, the process is prone to the existing races, and, more importantly, so is a new style process. Closes: #869546 [fixups: fix error messages, add public IsLocked() method, and make {Un,}LockInner return an error on !debSystem]
2018-06-13Fix lock counting in debSystemJulian Andres Klode
debSystem uses a reference counted lock, so you can acquire it multiple times in your applications, possibly nested. Nesting locks causes a fd leak, though, as we only increment the lock count when we already have locked twice, rather than once, and hence when we call lock the second time, instead of increasing the lock count, we open another lock fd. This fixes the code to check if we have locked at all (> 0). There is no practical problem here aside from the fd leak, as closing the new fd releases the lock on the old one due to the weird semantics of fcntl locks.
2018-05-11Start pkg records for deb files with dpkg outputDavid Kalnischkies
It is easier to prepend our fields, but that results in confusion for things working on the so generated records as they don't start with the usual "Package" – that shouldn't be a problem in theory, but in practice e.g. "apt-cache show" shows these records directly to the user who will probably be more confused by it than tools.
2018-05-07Remove obsolete RCS keywordsGuillem Jover
Prompted-by: Jakub Wilk <jwilk@debian.org>
2018-05-05Fix various typos reported by spellcheckersDavid Kalnischkies
Reported-By: codespell & spellintian Gbp-Dch: Ignore
2018-02-19Check that Date of Release file is not in the futureJulian Andres Klode
By restricting the Date field to be in the past, an attacker cannot just create a repository from the future that would be accepted as a valid update for a repository. This check can be disabled by Acquire::Check-Date set to false. This will also disable Check-Valid-Until and any future date related checking, if any - the option means: "my computers date cannot be trusted." Modify the tests to allow repositories to be up to 10 hours in the future, so we can keep using hours there to simulate time changes.
2018-01-17Introduce inrelease-path option for sources.listJulian Andres Klode
Allow specifying an alternative path to the InRelease file, so you can have multiple versions of a repository, for example. Enabling this option disables fallback to Release and Release.gpg, so setting it to InRelease can be used to ensure that only that will be tried. We add two test cases: One for checking that it works, and another for checking that the fallback does not happen. Closes: #886745
2018-01-05dpkg status parsing: check if name is valid before useDavid Kalnischkies
The summary line sounds a bit much: what we end up doing is just adding two more guards before using results which should always be valid™. That these values aren't valid is likely a bug in itself somewhere, but that is no reason for crashing.
2017-12-13give the methods more metadata about the files to acquireDavid Kalnischkies
We have quite a bit of metadata available for the files we acquire, but the methods weren't told about it and got just the URI. That is indeed fine for most, but to avoid methods trying to parse the metadata out of the provided URIs (and fail horribly in edgecases) we can just as well be nice and tell them stuff directly.
2017-12-13convert various c-style casts to C++-styleDavid Kalnischkies
gcc was warning about ignored type qualifiers for all of them due to the last 'const', so dropping that and converting to static_cast in the process removes the here harmless warning to avoid hidden real issues in them later on. Reported-By: gcc Gbp-Dch: Ignore
2017-12-13deprecate the single-line deprecation ignoring macroDavid Kalnischkies
gcc has problems understanding this construct and additionally thinks it would produce multiple lines and stuff, so to keep using it isn't really worth it for the few instances we have: We can just write the long form there which works better. Reported-By: gcc Gbp-Dch: Ignore
2017-12-13fix over-calculating dpkg commandline lengthDavid Kalnischkies
Mostly harmless as it just means that apt thinks that the dpkg commandline it is building is slightly longer than it actually is and we have various ways of avoiding generating very long lines nowadays, but calculating the right value can't hurt. Reported-By: gcc -Wmultistatement-macros
2017-09-26use pkgTagSection::Key in srcRecords parserDavid Kalnischkies
Using hardcoded array-indexes in the build-dependency parsing is efficient, but less discoverable and easier to break. We can avoid this by making it even more efficient (not that it would be noticeable) allowing us to do explicitly named comparisons instead. Gbp-Dch: Ignore
2017-09-26allow empty build-dependency fields in the parserDavid Kalnischkies
APT used to parse only wellformed files produced by repository creation tools which removed empty files as pointless before apt would see them. Now that apt can be told to parse e.g. debian/control files directly, it needs to be a little more accepting through: We had this with comments already, now let it deal with the far more trivial empty fields. Closes: #875363
2017-09-13Fix translator comment location for legacy target warningJulian Andres Klode
In commit Do not warn about duplicate "legacy" targets, we we added an if, that changed the .po files...
2017-09-09Do not warn about duplicate "legacy" targetsJulian Andres Klode
If a source has a legacy Contents file, and two lines mention the same archive but different components, a warning would be issued that is confusing. So, as the field is named Contents-deb-legacy, let's just not print warnings for fields containing "legacy". LP: #1697120 Closes: #839259
2017-09-09drop unused/unimplemented & hidden LoadReleaseInfoDavid Kalnischkies
The relevant calling code as well as the implementation for the deb system was removed 2 years ago with the refactoring of release information storage (b07aeb1a6e24825e534167a737043441e871de9f). This commit removes the the unused remains of this change with no practical effect on anybody (expect codesize) as the methods were declared as hidden and hence only libapt could have called it. Gbp-Dch: Ignore
2017-09-09don't write & chmod /dev/null log filesDavid Kalnischkies
APT by default logs terminal (term.log) and actions (history.log), but if either or Dir::Log directly is set to /dev/null it continues to do so, which isn't too bad – just wasted effort – but term.log is chmodded to protect it from the general public (as it may contain otherwise private data the admin entired in the terminal) which shouldn't happen for /dev/null.
2017-08-24Replace APT_CONST with APT_PURE everywhereJulian Andres Klode
As a follow up to the last commit, let's replace APT_CONST with APT_PURE everywhere to clean stuff up.
2017-07-12Reformat and sort all includes with clang-formatJulian Andres Klode
This makes it easier to see which headers includes what. The changes were done by running git grep -l '#\s*include' \ | grep -E '.(cc|h)$' \ | xargs sed -i -E 's/(^\s*)#(\s*)include/\1#\2 include/' To modify all include lines by adding a space, and then running ./git-clang-format.sh.
2017-07-12Drop cacheiterators.h includeJulian Andres Klode
Including cacheiterators.h before pkgcache.h fails because pkgcache.h depends on cacheiterators.h.
2017-07-12Handle supported components with slashes in sources.listApollon Oikonomopoulos
Commit d7c92411dc1f4c6be098d1425f9c1c075e0c2154 parses the Components section of (In)Release and attempts to detect the distribution's supported components. While doing so, it handles component names with slashes in a special manner, assuming that the actual component is only the part after the final slash. This is done to handle security.debian.org, which usually appears in sources.list as follows: deb http://s.d.o/debian-security stretch/updates main contrib non-free while the actual release file has: Codename: stretch Components: updates/main updates/contrib updates/non-free While this special handing on APTs part indeed works for debian-security, it emits spurious warnings on repositories that actually use slashes in the component names *and* appear so in sources.list. We fix this by adding both component versions (whole and final part) to the SupportedComponents array. Closes: #868127
2017-06-28Strip 0: epochs from the version hashJulian Andres Klode
This should fix some issues with dpkg normalizing such values. Suprisingly enough apt treats the Version: field the same, even with epoch vs without, but not when searching, and does not strip the 0: from the output.
2017-06-28show a Release-Notes URI if infos were changedDavid Kalnischkies
This gives the repository owner a chance to explain why this change was needed – e.g. explaining the organisational changes or simply detailing the changes in the new release made. Note that this URI is also shown if the change is accepted, so it also draws attention to release notes of minor updates (if users watch apt output closely).
2017-06-28error in update on Release information changesDavid Kalnischkies
The value of Origin, Label, Codename and co can be used in user configuration from apts own pinning to unattended upgrades. A repository changing this values can therefore have serious effects on the behaviour of apt and other tools using these values. In a first step we will generate error messages for these changes now explaining the need for explicit confirmation and provide config options and commandline flags to accept them.
2017-06-26fix some unlikely memory leaks in error casesDavid Kalnischkies
The error cases are just as unlikely as the memory leaks to ever cause real problems, but lets play it safe for correctness. Reported-By: scan-build & clang Gbp-Dch: Ignore
2017-06-26warn if an expected file can't be acquiredDavid Kalnischkies
If we couldn't find an entry for a Sources file we would generate an error while for a Packages file we would silently skip it due to assuming it is missing because it is empty. We can do better by checking if the repository declares that it supports a component we want to get the file from and if not say so and hint at the user making a typo. An example were this helps is mozilla.debian.net which dropped the firefox-aurora component (as upstream did) meaning no upgrades until the user notices manually that the repository doesn't provide packages anymore. With this commit warnings are raised hopefully causing the user to investigate what is wrong (sooner).
2017-05-31Fix parsing of or groups in build-deps with ignored packagesJulian Andres Klode
If the last alternative(s) of an Or group is ignored, because it does not match an architecture list, we would end up keeping the or flag, effectively making the next AND an OR. For example, when parsing (on amd64): debhelper (>= 9), libnacl-dev [amd64] | libnacl-dev [i386] => debhelper (>= 9), libnacl-dev | Which can cause python-apt to crash. Even worse: debhelper (>= 9), libnacl-dev [amd64] | libnacl-dev [i386], foobar => debhelper (>= 9), libnacl-dev [amd64] | foobar By setting the previous alternatives Or flag to the current Or flag if the current alternative is ignored, we solve the issue. LP: #1694697
2017-02-10Do not package names representing .dsc/.deb/... filesJulian Andres Klode
In the case of build-dep and other commands where a file can be passed we must make sure not to normalize the path name as that can have odd side effects, or well, cause the operation to do nothing. Test for build-dep-file is adjusted to perform the vcard check once as "vcard" and once as "VCard", thus testing that this solves the reported bug. We inline the std::transform() and optimize it a bit to not write anything in the common case (package names are defined to be lowercase, the whole transformation is just for names that should not exist...) to counter the performance hit of the added find() call (it's about 0.15% more instructions than with the existing transform, but we save about 0.67% in writes...). Closes: #854794
2017-01-19fix various typos reported by spellintianDavid Kalnischkies
Most of them in (old) code comments. The two instances of user visible string changes the po files of the manpages are fixed up as well. Gbp-Dch: Ignore Reported-By: spellintian
2017-01-19fix various typos reported by codespellDavid Kalnischkies
Nothing in user visible strings. Gbp-Dch: Ignore Reported-By: codespell
2017-01-02ParseDepends: Support passing the desired architectureNiels Thykier
This is useful for e.g. Britney, where the Build-Depends would have to be parsed for multiple architectures. With this change, the call can choose the architecture without having to mess with the config. Signed-off-by: Niels Thykier <niels@thykier.net> Closes: #845969 (jak@d.o: made the code compile)
2016-12-31fix minimum pkgs option for dpkg --recursive usageDavid Kalnischkies
Interpreting a boolean as an int works just fine – it just hasn't the intended result – it isn't a serious problem through as the disabling of the usage of this dpkg calling style is just an "optimization"
2016-12-31avoid producing invalid options if repo has no hostDavid Kalnischkies
This can happen e.g. for file: repositories. There is no inherent problem with setting such values internally, but its bad style, forbidden in the manpage and could be annoying in the future. Gbp-Dch: Ignore
2016-12-31gets file location via FindFile instead of manual mergeDavid Kalnischkies
Unlikely to have any practical effect, but its more consistent to use the right methods instead of performing it slightly incorrect by hand. Gbp-Dch: Ignore
2016-11-24skip unconfigure for unconfigured to-be removed pkgsDavid Kalnischkies
2016-11-24do not configure unconfigured to be removed packagesDavid Kalnischkies
We try to configure all packages at the end which need to be configured, but that also applies to packages which weren't completely installed (e.g. maintainerscript failed) we end up removing in this interaction instead. APT doesn't perform this explicit configure in the end as it is using "dpkg --configure --pending", but it does confuse the progress report and potentially also hook scripts. Regression-Of: 9ffbac99e52c91182ed8ff8678a994626b194e69
2016-11-24don't perform implicit crossgrades involving M-A:sameDavid Kalnischkies
dpkg stumbles over these (#844300) and we haven't dropped 'easier' removes to be implicit and to be scheduled by dpkg by default so far so we shouldn't push the decision in such cases to dpkg either.
2016-11-24improve arch-unqualified dpkg-progress parsingDavid Kalnischkies
Our old idea was to look for the first package which would be "touched" and take this as the package dpkg is talking about, but that is incorrect in complicated situations like a package upgraded to/from multiple M-A:same siblings installed. As we us the progress report to decide what is still needed we have to be reasonabily right about the package dpkg is talking about, so we jump to quite a few loops to get it.
2016-11-23correct cross & disappear progress detectionDavid Kalnischkies
Given that we use the progress information to skip over actions dpkg has already done like not purging a package which was already removed and had no config files or not acting on disappeared packages and such it is important that apt and dpkg agree on which states the package has to pass through. To ensure that we keep tabs on this in the future a warning is added at the end if apt hasn't seen all the action it was supposed to see. I can't wait for the first bugreporters to wonder about this…
2016-11-23react to trig-pend only if we have nothing else to doDavid Kalnischkies
If a package is triggered dpkg frequently issues two messages about it causing us to make a note about it both times which messes up our planned dpkg actions view. Adding these actions if we have nothing else planned fixes this and should still be correct as those planned actions will deal with the triggering just fine and we avoid strange problems like a package triggered before its removed…
2016-11-22Do not use MD5SumValue for Description_md5()Julian Andres Klode
Our profile says we spend about 5% of the time transforming the hex digits into the binary format used by HashsumValue, all for comparing them against the other strings. That makes no sense at all. According to callgrind, this reduces the overall instruction count from 5,3 billion to 5 billion in my example, which roughly matches the 5%.
2016-11-22debListParser: Micro-optimize AvailableDescriptionLanguages()Julian Andres Klode
Generating a string for each version we see is somewhat inefficient. The problem here is that the Description tag names are longer than 15 byte, and thus require an allocation on the heap, which we should avoid. It seems reasonable that 20 characters works for all languages codes used for archive descriptions, but if not, there's a warning, so we'll catch that. This should improve performance by about 2%.
2016-11-22Optimize VersionHash() to not need temporary copy of inputJulian Andres Klode
Stop copying stuff, and just parse the bytes one by-one to the newly created AddCRC16Byte. This improves the instruction count for an update run from 720,850,121 to 455,801,749 according to callgrind.
2016-11-22Introduce tolower_ascii_unsafe() and use it for hashingJulian Andres Klode
This one has some obvious collisions for non-alphabetical characters, like some control characters also hashing to numbers, but we don't really have those, and these are hash functions which are not collision free to begin with.