summaryrefslogtreecommitdiff
path: root/test/integration
AgeCommit message (Collapse)Author
2016-01-08keep compressed indexes in a low-cost formatDavid Kalnischkies
Downloading and storing are two different operations were different compression types can be preferred. For downloading we provide the choice via Acquire::CompressionTypes::Order as there is a choice to be made between download size and speed – and limited by whats available in the repository. Storage on the other hand has all compressions currently supported by apt available and to reduce runtime of tools accessing these files the compression type should be a low-cost format in terms of decompression. apt traditionally stores its indexes uncompressed on disk, but has options to keep them compressed. Now that apt downloads additional files we also deal with files which simply can't be stored uncompressed as they are just too big (like Contents for apt-file). Traditionally they are downloaded in a low-cost format (gz) as repositories do not provide other formats, but there might be even lower-cost formats and for download we could introduce higher-cost in the repositories. Downloading an entire index potentially requires recompression to another format, so an update takes potentially longer – but big files are usually updated via pdiffs which has to de- and re-compress anyhow and does it on the fly anyhow, so there is no extra time needed and in general it seems to be benefitial to invest the time in update to save time later on file access.
2016-01-08tests: try to pick up compressors from config automaticallyDavid Kalnischkies
Less hardcoding should help while introducing new compressors. Git-Dch: Ignore
2016-01-08allow pdiff bootstrap from all supported compressorsDavid Kalnischkies
There is no reason to enforce that the file we start the bootstrap with is compressed with a compressor which is available online. This allows us to change the on-disk format as well as deals with repositories adding/removing support for a specific compressor.
2016-01-08use one 'store' method to rule all (de)compressorsDavid Kalnischkies
Adding a new compressor method meant adding a new method as well – even if that boilt down to just linking to our generalized decompressor with a new name. That is unneeded busywork if we can instead just call the generalized decompressor and let it figure out which compressor to use based on the filenames rather than by program name. For compatibility we ship still 'gzip', 'bzip2' and co, but they are just links to our "new" 'store' method.
2016-01-03test-apt-cache: Adjust for hashtable size changeJulian Andres Klode
Gbp-Dch: ignore
2016-01-02Add new APT::Keep-Downloaded-Packages optionMichael Vogt
This option controls if downloaded packages should be kept after a successful install or if they should be deleted. The default for "apt-get" is that they are kept (just like before). However the default for "apt" is that they get deleted. Closes: #160743
2016-01-02properly parse comments in apt_preferences and deb822-style sourcesDavid Kalnischkies
apt_preferences and deb822-style sources used the specialized class pkgUserTagSection to deal with comments before/after a given stanza, but it couldn't deal with comments in the stanza at all. codesearch suggests that nobody else does and a vastely superior way of working with potentially commented files is implemented now, so we can officially discourage the use of the old incomplete hack class.
2016-01-02support comments in debian/control parsingDavid Kalnischkies
Now (55153bf94ff28a23318e79aa48242244c4d82b3c) that pkgTagFile can be told to deal with all sorts of comments we can use this mode to parse dsc (as by catch) and debian/control files properly even in the wake of multiline fields spliced with comments like Build-Depends. Closes: 806775
2015-12-29Fix test case that was broken by switch of hash functionJulian Andres Klode
This test relies on the ordering of the hash function.
2015-12-27allow repositories to forbid arch:all for specific index targetsDavid Kalnischkies
Debian has a Packages file for arch:all already, but the arch:any files contain arch:all packages as well, so downloading it would be a total waste of resources. Getting this solved is on the list of things to do, but it is also the hardest part – for index targets like Contents the situation is much easier and less server/client implementations are involved so we might not want to stall them. A repository can now declare via: No-Support-for-Architecture-all: Packages that even if an arch:all Packages exists, it shouldn't be downloaded, so that support for Contents files can be added now. See also 1dd20368486820efb6ef4476ad739e967174bec4 for the implementation of downloading arch:all index targets, which this is limiting. The field uses the name of the target from the apt configuration for simplicity and is negative by design as this field is intended to be supported/needed only for a "short" time (one or two Debian releases). While this commit theoretically supports any target, its expected to only see "Packages" as a value in reality.
2015-12-23ensure we got a lock in clean operationDavid Kalnischkies
We try to acquired the locks, but we didn't stop if we failed to get it… Closes: 808561
2015-12-19tests: support gpg2 properly in all testcasesDavid Kalnischkies
The output changes slightly between different versions, which we already dealt with in the main testcase for apt-key, but there are two more which do not test both versions explicitly and so still had gpg1 output to check against as this is the default at the moment. Git-Dch: Ignore
2015-12-19avoid evaluating shell in paths used in apt-keyDavid Kalnischkies
apt-key creates internally a script (since ~1.1) which it will call to avoid dealing with an array of different options in the code itself, but while writing this script it wraps the values in "", which will cause the shell to evaluate its content upon execution. To make 'use' of this either set a absolute gpg command or TMPDIR to something as interesting as: "/tmp/This is fü\$\$ing cràzy, \$(man man | head -n1 | cut -d' ' -f1)\$!" If such paths can be encountered in reality is a different question…
2015-12-19tests: support spaces in path and TMPDIRDavid Kalnischkies
This doesn't allow all tests to run cleanly, but it at least allows to write tests which could run successfully in such environments. Git-Dch: Ignore
2015-12-14test-apt-download-progress: Run the https test multiple timesJulian Andres Klode
This filters out errors due to timing issues. Early exits if enough pulses occured.
2015-12-14pkgcache: Make hash arch-independent using fixed size integerJulian Andres Klode
This helps writing test cases. Also adapt the test case that expected 64-bit. Nothing changes performance wise, the distribution of the hash values remains intact.
2015-12-14tagfile: Hardcode error message for out of range integer valuesJulian Andres Klode
This makes the test suite work on 32 bit-long platforms. Gbp-Dch: ignore
2015-12-14test framework: More noopchroot fixesJulian Andres Klode
Use asprintf() so we have easy error detection and do not depend on PATH_MAX. Do not add another separator to the generated path, in both cases the path inside the chroot is guaranteed to have a leading / already. Also pass -Wall to gcc.
2015-12-14test framework: Unset no_proxy as wellJulian Andres Klode
This caused test-bug-717891-abolute-uris-for-proxies to fail Gbp-Dch: ignore
2015-12-14test framework: Unset http proxy variables in setupenvironmentJulian Andres Klode
This breaks a lot of test cases Gbp-Dch: ignore
2015-12-14test framework: Correctly generate new paths in noopchrootJulian Andres Klode
The allocated buffer was one byte too small. Allocate a buffer of PATH_MAX instead and use snprintf(), as suggested by Martin Pitt.
2015-12-14test-apt-download-progress: Limit rate to 600kJulian Andres Klode
This reduces the chance that the test fails. Gbp-Dch: ignore
2015-12-14test-apt-download-progress: Allow smaller progress changeJulian Andres Klode
Instead of checking for [10%, 100%), check for (0%, 100%), that is everything < 100% and >0%. Gbp-Dch: ignore
2015-12-14test-sourceslist-lang-plusminus-options: Use amd64 architectureJulian Andres Klode
This should make the test work on non-amd64 systems Gbp-Dch: ignore
2015-12-14non-existing directories don't need to be cleanedDavid Kalnischkies
Trying to clean up directories which do not exist seems rather silly if you think about it, so let apt think about it and stop it. Depends a bit on the caller if this is fixing anything for them as they might try to acquire a lock or doing other clever things as apt does. Closes: 807477
2015-12-14support regex and co in 'apt-cache policy $pkg' againDavid Kalnischkies
Regression of 1e064088bf7b3e29cd36d30760fb3e4143a1a49a (1.1~exp4) which moved code around and renamed methods heavily ending up calling the wrong method matching packagenames only instead of calling the full array. Most commands work with versions, so this managed to fly under the radar for quite a while. Closes: 807870
2015-12-14show a more descriptive error for weak Release filesDavid Kalnischkies
If we can't work with the hashes we parsed from the Release file we display now an error message if the Release file includes only weak hashes instead of downloading the indexes and failing to verify them with "Hash Sum mismatch" even through the hashes didn't mismatch (they were just weak). If for some (unlikely) reason we have got weak hashes only for individual targets we will show a warning to this effect (again, befor downloading and failing the index itself). Closes: 806459
2015-12-11Convert package names from Packages files to lower caseJulian Andres Klode
dpkg does that when reading package files, so we should do the same. This only deals with parsing names from binary package paragraphs, it does not look at source package names and/or the list of binaries in a dsc file. Closes: #807012
2015-12-11test: Fix testcases relying on mixed case package namesJulian Andres Klode
Gbp-Dch: ignore
2015-12-06replace run-parts with find|sort to avoid debianutils usageDavid Kalnischkies
After e75e5879 the reason for an implicit dependency on debianutils (which is essential for debian, but likely not on other systems) was just two uses of run-parts, which can be replaced with the a lot more portable find-piped-into-sort duo.
2015-12-06replace "which" with "command -v" for portabilityDavid Kalnischkies
which is a debian specific tool packaged in debianutils (essential) while command is a shell builtin defined by POSIX. Closes: 807144 Thanks: Mingye Wang for the suggestion.
2015-12-02use @CHANGEPATH@ as placeholder in changelog URI templatesDavid Kalnischkies
This should make it more obvious that CHANGEPATH is a placeholder which apt will replace with a package specific path rather than a string constant. Mail-Reference: <87d1upgvaf.fsf@deep-thought.43-1.org> Mail-Archive: https://lists.debian.org/debian-dak/2015/12/msg00005.html
2015-12-02tests: apt build-dep actually reports no depends correctlyDavid Kalnischkies
'Regression' of 7d19ee92f2368a40e739cb27d22d6d28f37ebf45, just that it now works more as expected than previously. Of course, build-essentials are implicitly also build dependencies, so by definition all packages have build dependencies, but that isn't what this message wants to say and it isn't what the user expects. Git-Dch: Ignore
2015-12-01require explicit paths to dsc/control as we do for deb filesDavid Kalnischkies
Otherwise a user is subject to unexpected content-injection depending on which directory she happens to start apt in. This also cleans up the code requiring less implementation details in build-dep which is always good. Technically, this is an ABI break as we override virtual methods, but that they weren't overridden was a mistake resulting in pure classes, which shouldn't be pure, so they were unusable – and as they are new in 1.1 nobody is using them yet (and hopefully ever as they are borderline implementation details). Closes: 806693
2015-11-29accept ../ on the cmdline as start for a deb file as wellDavid Kalnischkies
Regression of 14341a7ee1ca3dbcdcdbe10ad19b947ce23d972d. Reported-By: Julian Andres Klode <jak@debian.org>
2015-11-29tests: fix typos, correct helpmsgs and test testsDavid Kalnischkies
Git-Dch: Ignore
2015-11-29do not override exact targetrelease matches with lesser matchesAndreas Cadhalpun
The relevant testcases are in test/integration/test-apt-get-source. There is a test for #731853 that is supposed to "ensure that apt will pick the higher version number" of 0.0.1 (stable) and 0.1 (stable). However, this works by pure chance, as simply reversing the order of the two insertsource lines makes the test fail. So #731853 isn't really fixed, yet. Actually, that's related to the problem I reported, as the underlying issue for both is the same: In the FindSrc function apt chooses a new 'best hit', if either * there is a target release and it matches the release of the package, * or the version of the package is higher than the last best hit. Consider having 1.0 (stable), 2.0 (unstable) and 1.5 (unstable), in this order. Looking for the version in stable, apt first selects 1.0, because the release matches the target release, but then subsequently selects 2.0, because the version is higher. Looking for the version in unstable, apt first selects 2.0, because the release matches the target release, but then subsequently selects 1.5, because the release also matches the target release. The correct way would be to choose a new 'best hit', if either * there is a target release and it matches the release of the package, * or there is no target release and the version is higher than the last best hit. Closes: 746412 Mail-Reference: <565A604B.7090104@googlemail.com> Mail-Archive: https://lists.debian.org/debian-devel/2015/11/msg00470.html
2015-11-28disable privilege-drop verification by default as fakeroot trips over itDavid Kalnischkies
Dropping privileges is an involved process for code and system alike so ideally we want to verify that all the work wasn't in vain. Stuff designed to sidestep the usual privilege checks like fakeroot (and its many alternatives) have their problem with this through, partly through missing wrapping (#806521), partly as e.g. regaining root from an unprivileged user is in their design. This commit therefore disables most of these checks by default so that apt runs fine again in a fakeroot environment. Closes: 806475
2015-11-28tests: use id to get user/group instead of environmentDavid Kalnischkies
debci seems to have a cleaner environment now and even if not we could never guess nogroup, so figure it out properly via 'id'. Git-Dch: Ignore
2015-11-25slightly rephrase notice shown for insecure repositoriesJustin B Rye
Git-Dch: Ignore
2015-11-21show potentially arch-qualified fullname in 'apt show'David Kalnischkies
We do not show the architecture as a dedicated field as this is rather technical information, but as packagename it makes sense to show the architecture as other part of apt will refer to it in this way.
2015-11-21review of new/changed translatable program stringsJustin B Rye
Reference mail: https://lists.debian.org/debian-l10n-english/2015/11/msg00006.html
2015-11-21do not sent Last-Modified if we expect a changed fileDavid Kalnischkies
In 8d041b4f we made apt figure out based on the last Release file it has if it should request a file or not given that the hashes changed or not. So if we have a last Release file and do a request, do not sent a Last-Modified header as we expect a change so much that a non-change would indeed be an error. The Last-Modified header is therefore at best ignored by the server, so sending it is just wasted effort. In the worst case as time is a fragile thing the server decides against sending us an update with the idea that we already have the latest content, which we know for a fact that we haven't. Given that we sent less information to the server our request is on its own also less identifiable as coming from a returning or new user. The disadvantage is that if we end up getting an old index file after getting a new Release file from another mirror the old mirror will not be able to tell us 'Hit', but instead sends us the complete file we discard, but both lets us end up with the same error class in the end, so the difference isn't big in practice.
2015-11-19ignore lost+found in private directory cleanupDavid Kalnischkies
In ce1f3a2c we started warning about failing unlinking, which we consistently do for directories. That isn't a problem as directories usually aren't in the places we do want to clean up – with the potential exeception of "lost+found", so lets ignore it like we ignore our own partial/ subdirectory. Closes: 805424
2015-11-19tests: use quiet level 0 by default in testsDavid Kalnischkies
Git-Dch: Ignore
2015-11-19support setting empty values (sanely) & removing support forDavid Kalnischkies
space-gapping: '-o option= value' That is a very old feature (straight from 1998), but it is super surprising if you try setting empty values and instead get error messages or a non-empty value as the next parameter is treated as the value – which could have been empty, so if for some reason you need a compatible way of setting an empty value try: '-o option="" ""'. I can only guess that the idea was to support '-o option value', but we survived 17 years without it, we will do fine in the future I guess. Similar is the case for '-t= testing' even through '-t testing' existed before and the code even tried to detect mistakes like '-t= -b' … all gone now. Technically that is as its removing a feature replacing it with another a major interface break. In practice I really hope for my and their sanity that nobody was using this; but if for some reaon you do: Remove the space and be done. I found the patch and the bugreport actually only after the fact, but its reassuring that others are puzzled by this as well and hence a thanks is in perfect order here as the patch is practical identical [expect that this one here adds tests and other bonus items]. Thanks: Daniel Hartwig for initial patch. Closes: 693092
2015-11-19do not use _apt for file/copy sources if it isn't world-accessibleDavid Kalnischkies
In 0940230d we started dropping privileges for file (and a bit later for copy, too) with the intend of uniforming this for all methods. The commit message says that the source will likely fail based on the compressors already – and there isn't much secret in the repository content. After all, after apt has run the update everyone can access the content via apt anyway… There are sources through which worked before which are mostly single-deb (and those with the uncompressed files available). The first one being especially surprising for users maybe, so instead of failing, we make it so that apt detects that it can't access a source as _apt and if so doesn't drop (for all sources!) privileges – but we limit this to file/copy, so the uncompress which might be needed will still fail – but that failed before this regression. We display a notice about this, mostly so that if it still fails (e.g. compressed) the user has some idea what is wrong. Closes: 805069
2015-11-18tests: fail testsuccess if notices are shown, tooDavid Kalnischkies
Notices are just hints, but if they are printed in tests, they should be expected and if not the test should fail. No current test has this problem, so that is just potential future proving. Git-Dch: Ignore
2015-11-05"support" unsigned Release files without hashes againDavid Kalnischkies
This 'ignores' the component Release files you can find in Debian alongside the binary-* directories, which isn't exactly a common usecase, but it worked before, so lets support it again as this isn't worse than a valid Release file which is unsigned. Git-Dch: Ignore
2015-11-05allow acquire method specific options via Binary scopeDavid Kalnischkies
Allows users who know what they are getting themselves into with this trick to e.g. disable privilege dropping for e.g. file:// until they can fix up the permissions on those repositories. It helps also the test framework and people with a similar setup (= me) to run in less modified environments.