summaryrefslogtreecommitdiff
path: root/apt-pkg/deb
AgeCommit message (Collapse)Author
2021-01-08Add support for Phased-Update-PercentageJulian Andres Klode
This adds support for Phased-Update-Percentage by pinning upgrades that are not to be installed down to 1. The output of policy has been changed to add the level of phasing, and documentation has been improved to document how phased updates work. The patch detects if it is running in a chroot, and if so, always includes phased updates, restoring classic apt behavior to avoid behavioral changes on buildd chroots. Various options are added to control this all: * APT::Get::{Always,Never}-Include-Phased-Updates and their legacy update-manager equivalents to always or never include phased updates * APT::Machine-ID can be set to a UUID string to have all machines in a fleet phase the same * Dir::Etc::Machine-ID is weird in that it's default is sort of like ../machine-id, but not really, as ../machine-id would look up $PWD/../machine-id and not relative to Dir::Etc; but it allows you to override the path to machine-id (as opposed to the value) * Dir::Bin::ischroot is the path to the ischroot(1) binary which is used to detect whether we are running in a chroot.
2021-01-04Determine autoremovable kernels at run-timeJulian Andres Klode
Our kernel autoremoval helper script protects the currently booted kernel, but it only runs whenever we install or remove a kernel, causing it to protect the kernel that was booted at that point in time, which is not necessarily the same kernel as the one that is running right now. Reimplement the logic in C++ such that we can calculate it at run-time: Provide a function to produce a regular expression that matches all kernels that need protecting, and by changing the default root set function in the DepCache to make use of that expression. Note that the code groups the kernels by versions as before, and then marks all kernel packages with the same version. This optimized version inserts a virtual package $kernel into the cache when building it to avoid having to iterate over all packages in the cache to find the installed ones, significantly improving performance at a minor cost when building the cache. LP: #1615381
2020-12-18Keep URIs encoded in the acquire systemDavid Kalnischkies
We do not deal a lot with URIs which need encoding, but then we do it is a pain that we store it decoded in the acquire system as it means we have to decode and reencode URIs eventually which is potentially giving us slightly different URIs. We see that in our own testing framework while setting up redirects as the config options are effectively double-encoded and decoded to pass them around successfully as otherwise %2f and / in an URI are treated the same. This commit adds the infrastructure for methods to opt into getting URIs send in encoded form (and returning them to us in encoded form, too) so that we eventually do not have to touch the URIs which is how it should be. This means though that we have to deal with methods who do not support this yet (aka: all at the moment) for which we decode and encode while communicating with them.
2020-12-09CVE-2020-27350: debfile: integer overflow: Limit control size to 64 MiBJulian Andres Klode
Like the code in arfile.cc, MemControlExtract also has buffer overflows, in code allocating memory for parsing control files. Specify an upper limit of 64 MiB for control files to both protect against the Size overflowing (we allocate Size + 2 bytes), and protect a bit against control files consisting only of zeroes.
2020-06-29Add basic support for the Protected fieldJulian Andres Klode
This will be mapped to Important for the time being.
2020-06-14Do not sent our filename-provides trick to EDSP solversDavid Kalnischkies
If package is installed via an explicitly given deb file we store the filename as a provides, so that the frontend can request the filename and get the usual "Selected foo instead of foo.deb" message. We do not need to trouble the EDSP solvers with that though as these provides are not valid in various ways and we have already solved the link between commandline and package (and version) for them. Closes: #962741
2020-05-25Silence clang warning -Wstring-plus-intDavid Kalnischkies
../apt-pkg/init.cc:137:39: warning: adding 'int' to a string does not append to the string [-Wstring-plus-int] Cnf.CndSet("Dir::State", STATE_DIR + 1); ../apt-pkg/init.cc:137:39: note: use array indexing to silence this warning We have a few instances of that & it should be reasonably clear that we are not actually trying to append here, but ignoring or silencing this warning with an override is far more costly than just using what clang suggests here. Reported-By: clang Gbp-Dch: Ignore
2020-03-21Parse last line in deb file correctly by adding a newlineDavid Kalnischkies
While merging apt-pkg and apt-inst libraries the codepath of handling deb files in apt-pkg was adapted to use the 'old' code from apt-inst instead of fork&exec of dpkg-deb -I. The information we get this way forms the main part of the package stanza, but we add a few semi-optional fields to the stanza to make it look and work more like a stanza we got from a repository. Just be careful with the area where these two parts touch as if, hypothetically, we would stip all newlines around the parts, but forget to add a newline between them later, the two lines around the merge would stick a bit too close together forming one which could result in fun parsing errors if this merged line was previously e.g. a well-formed Depends line and has now extra fluff attached. This codepath has a history with too many newlines (#802553) though, so how likely is it really that it will some day lack one you may ask. References: 6089a4b17c61ef30b2efc00e270b0907f51f352a
2020-03-06Show absolute time while waiting for lock instead of %, rework messageJulian Andres Klode
Showing a percentage for a timeout is pretty non-standard. Rework the progress class so it can show an absolute progress (currently hardcoded to use seconds as a unit). If there is a timeout (aka if it's not the maximum long long unsigned -1llu), then show the timeout, otherwise just count up seconds, e.g. Waiting for cache lock: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 33842 (apt)... 1/120s or Waiting for cache lock: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 33842 (apt)... 1s Also improve the error message to use "Waiting for cache lock: %s" instead of "... (%s)", as having multiple sentences inside parenthesis is super weird, as is having two closing parens. We pass the information via _config, as that's reasonably easy and avoids ABI hackage. It also provides an interesting debugging tool for other kinds of progress.
2020-02-27(temporarily) unhide pkgDPkgPM again to have python-apt compileJulian Andres Klode
2020-02-26apt(8): Wait for frontend and cache lockJulian Andres Klode
This is a rework of !6 with additional stuff for the frontend lock, so we can lock the frontend lock and then keep looping over dpkg lock.
2020-02-26apt-pkg: default visibility to hiddenJulian Andres Klode
2020-02-26Make metaIndex::GetNotBefore virtualJulian Andres Klode
2020-02-26pkgsystem: Drop more virtual workaround shenanigansJulian Andres Klode
2020-02-26metaindex: Add Origin, Label, Version, DefaultPin, ReleaseNotes membersJulian Andres Klode
These were hidden behind the d-pointer previously.
2020-02-26Remove various dynamic_cast uses, use virtual methods insteadJulian Andres Klode
2020-02-26Remove ABI workaround for debDebPkgFileIndex::ArchiveInfoJulian Andres Klode
2020-02-24Make map_pointer<T> typesafeJulian Andres Klode
Instead of just using uint32_t, which would allow you to assign e.g. a map_pointer<Version> to a map_pointer<Package>, use our own smarter struct that has strict type checking. We allow creating a map_pointer from a nullptr, and we allow comparing map_pointer to nullptr, which also deals with comparisons against 0 which are often used, as 0 will be implictly converted to nullptr.
2020-02-18Remove CRC-16 implementationJulian Andres Klode
2020-02-18Use a 32-bit djb VersionHash instead of CRC-16Julian Andres Klode
2020-02-18Merge branch 'pu/source-version' into 'master'Julian Andres Klode
ABI BREAK: Implement pinning by source package See merge request apt-team/apt!96
2020-02-18Remove code tagged APT_PKG_590, add some missing includesJulian Andres Klode
Remove all code scheduled to be removed after 5.90, and fix files to include files they previously got from hashes.h including more headers.
2020-02-16Revert "Add a Packages-Require-Authorization Release file field"Julian Andres Klode
This experiment did not turn out sensibly, as some servers do not accept credentials when none are expected and fail, so you cannot mirror such a repository. This reverts commit c2b9b0489538fed4770515bd8853a960b13a2618.
2020-01-17Allow querying all binaries built by a source packageJulian Andres Klode
This adds a simple way to lookup binaries by a source package, but this adds all binaries into one list, even with different source versions. Be careful.
2020-01-14Remove includes of (md5|sha1|sha2).h headersJulian Andres Klode
Remove it everywhere, except where it is still needed.
2020-01-14Convert users of {MD5,SHA1,SHA256,SHA512}Summation to use HashesJulian Andres Klode
This makes use of the a function GetHashString() that returns the specific hash string. We also need to implement another overload of Add() for signed chars with sizes, so the existing users do not require reinterpret_cast everywhere.
2020-01-08Avoid extra out-of-cache hash table deduplication for package namesJulian Andres Klode
We were de-duplicating package name strings in StoreString, but also deduplicating most of them by them being in groups, so we had extra hash table lookups that could be avoided in NewGroup(). To continue deduplicating names across binary packages and source packages, insert groups for source packages as well. This is also a good first step in allowing efficient lookup of packages by source package - we can extend Group later by a list of SourceVersion objects, or alternatively, simply add a by-source chain into pkgCache::Version. This change improves performance by about 10% (913 to 814 ms), while having no significant overhead on the cache size: --- before +++ after @@ -1,7 +1,7 @@ -Total package names: 109536 (2.191 k) -Total package structures: 118689 (4.748 k) +Total package names: 119642 (2.393 k) +Total package structures: 118687 (4.747 k) Normal packages: 83309 - Pure virtual packages: 3365 + Pure virtual packages: 3363 Single virtual packages: 17811 Mixed virtual packages: 1973 Missing: 12231 @@ -10,21 +10,21 @@ Total distinct descriptions: 149291 (3.583 k) Total dependencies: 484135/156650 (12,2 M) Total ver/file relations: 57421 (1.378 k) Total Desc/File relations: 18219 (437 k) -Total Provides mappings: 29963 (719 k) +Total Provides mappings: 29959 (719 k) Total globbed strings: 226993 (5.332 k) Total slack space: 26,8 k -Total space accounted for: 38,1 M +Total space accounted for: 38,3 M Total buckets in PkgHashTable: 50503 - Unused: 5727 - Used: 44776 - Utilization: 88.6601% - Average entries: 2.65073 + Unused: 5728 + Used: 44775 + Utilization: 88.6581% + Average entries: 2.65074 Longest: 60 Shortest: 1 Total buckets in GrpHashTable: 50503 - Unused: 5727 - Used: 44776 - Utilization: 88.6601% - Average entries: 2.44631 - Longest: 10 + Unused: 4649 + Used: 45854 + Utilization: 90.7946% + Average entries: 2.60919 + Longest: 11 Shortest: 1
2019-09-19Pass --abort-after=1 to dpkg when using --force-dependsJulian Andres Klode
Using --force-depends causes dpkg to continue removing packages a package depends upon even if that package fails to be removed, because dpkg turns off all sanity checks. So we gotta tell dpkg to stop immediately if there's an error removing stuff. Closes: #935910 LP: #1844634
2019-07-08Apply various suggestions by cppcheckDavid Kalnischkies
Reported-By: cppcheck
2019-07-08Show details about the package with bad ProvidesDavid Kalnischkies
The error messages say only which package it was trying to provide, but not which package & version tried it which can be misleading as to which package (version) is the offender. References: #930256
2019-06-17RFC1123StrToTime: Accept const std::string& as first argumentJulian Andres Klode
We are converting to std::string anyway by passing to istringstream, and this removes the need for .c_str() in callers.
2019-06-12Run unifdef -DAPT_{8,9,10,15}_CLEANER_HEADERSJulian Andres Klode
2019-06-11Do not include squashfs file systems in df output.Brian Murray
LP: #1756595 Fixes Debian/apt#94
2019-06-11Merge the ParseDepends functionsJulian Andres Klode
2019-06-11Make APT::StringView publicJulian Andres Klode
2019-06-11Merge tag '1.8.2'Julian Andres Klode
apt Debian release 1.8.2
2019-05-21Unlock dpkg locks in reverse locking orderJulian Andres Klode
We need to unlock in the reverse order of locking in order to get useful behavior. LP: #1829860
2019-05-07debmetaindex: Use isspace_ascii() variant to normalize Signed-ByJulian Andres Klode
Using the locale-dependent isspace() function here opens us up to strange locale-dependent behavior.
2019-05-06CMake: Enforce "override" use on overriden methodsJulian Andres Klode
This ensures that we do not accidentally stop overriding a method because it's signature changed in an API break.
2019-05-06Use debDebFile to get control file instead of dpkg-debJulian Andres Klode
2019-05-06Merge libapt-inst into libapt-pkgJulian Andres Klode
2019-05-03Prevent shutdown while running dpkgJulian Andres Klode
As long as we are running dpkg, keep an inhibitor that blocks us from shutting down. LP: #1820886
2019-05-03Prevent shutdown while running dpkgJulian Andres Klode
As long as we are running dpkg, keep an inhibitor that blocks us from shutting down. LP: #1820886
2019-04-30Add 'explicit' to most single argument constructorsJulian Andres Klode
This prevents implicit conversions that we do not want, such as having a FileFd* being converted to a debListParser. Two cases are not yet handled because they require changes in code using them: 1. The classes in hashes.h 2. The URI class - this one is used quite a lot
2019-02-26dpkgpm: Remove deprecated pkgDPkgPM::SendV2Pkgs()Julian Andres Klode
2019-02-26packagemanager: Remove deprecated functionsJulian Andres Klode
2019-02-26metaindex: Remove deprecated checkSum::Hash (replaced by Hashes)Julian Andres Klode
2019-02-26pkgSrcRecords::Parser: Fold Files2() into Files()Julian Andres Klode
This is possible now with the API break. Cleaner code, woohoo.
2019-02-04Merge branch 'pu/dead-pin' into 'master'Julian Andres Klode
A pin of -32768 overrides any other, disables repo See merge request apt-team/apt!40
2019-02-01Add a Packages-Require-Authorization Release file fieldJulian Andres Klode
This new field allows a repository to declare that access to packages requires authorization. The current implementation will set the pin to -32768 if no authorization has been provided in the auth.conf(.d) files. This implementation is suboptimal in two aspects: (1) A repository should behave more like NotSource repositories (2) We only have the host name for the repository, we cannot use paths yet. - We can fix those after an ABI break. The code also adds a check to acquire-item.cc to not use the specified repository as a download source, mimicking NotSource.