summaryrefslogtreecommitdiff
path: root/test/integration/framework
AgeCommit message (Collapse)Author
2018-09-11Unset more environment variables in test frameworkDavid Kalnischkies
It is an uphill battle to "reset" the environment to a clean state without making it needlessly hard to use 'good' environment variables, so we just try a little harder here without really trying for completeness. Gbp-Dch: Ignore
2018-05-28Don't show acquire warning for "hidden" componentsDavid Kalnischkies
Commit d7c92411dc1f4c6be098d1425f9c1c075e0c2154 introduced a warning for non-existent files from components not mentioned in Components to hint users at a mispelling or the disappearance of a component. The debian-installer subcomponent isn't actively advertised in the Release file through, so if apt ends up in acquiring a file which doesn't exist for this component (like Translation files) apt would produce a warning: W: Skipping acquire of configured file 'main/debian-installer/i18n/Translation-en' as repository 'http://deb.debian.org/debian buster InRelease' doesn't have the component 'main/debian-installer' (component misspelt in sources.list?) We prevent this in the future by checking if any file exists from this component which results in the warning to be produced still for the intended cases and silence it on the d-i case. This could potentially cause the warning not to be produced in cases it should be if some marginal file remains, but as this message is just a hint and the setup a bit pathological lets ignore it for now. There is also the possibility of having no file present as they would all be 0-length files and being a "hidden" component, but that would be easy to workaround from the repository side and isn't really actively used at the moment in the wild. Closes: #879591
2018-05-28tests: Prevent stunnel4 from binding on IPv6David Kalnischkies
Hardcoding the IPv4 address 127.0.0.1 stops stunnel4 from also binding on IPv6 as well which not only binds on another port but confuses our crude port extraction by splitting on ':' with ::1. Gbp-Dch: Ignore
2018-05-05Fix various typos reported by spellcheckersDavid Kalnischkies
Reported-By: codespell & spellintian Gbp-Dch: Ignore
2018-04-15Merge branch 'pu/zstd' into 'master'Julian Andres Klode
pu/zstd See merge request apt-team/apt!8
2018-04-06test: export GCOV_ERROR_FILE=/dev/null to make it fail less/no testsJulian Andres Klode
2018-03-12apt-pkg: Add support for zstdJulian Andres Klode
zstd is a compression algorithm developed by facebook. At level 19, it is about 6% worse in size than xz -6, but decompression is multiple times faster, saving about 40% install time, especially with eatmydata on cloud instances.
2018-02-19Merge branch 'pu/not-valid-before' into 'master'Julian Andres Klode
Check that Date of Release file is not in the future See merge request apt-team/apt!3
2018-02-19Check that Date of Release file is not in the futureJulian Andres Klode
By restricting the Date field to be in the past, an attacker cannot just create a repository from the future that would be accepted as a valid update for a repository. This check can be disabled by Acquire::Check-Date set to false. This will also disable Check-Valid-Until and any future date related checking, if any - the option means: "my computers date cannot be trusted." Modify the tests to allow repositories to be up to 10 hours in the future, so we can keep using hours there to simulate time changes.
2018-02-19tests: set debhelper compat 10 and R³ by defaultDavid Kalnischkies
The testpackages hardly need debhelper at all, so any version would do, and they build without root rights by definition, but declaring it explicitly can't hurt and in the case of debhelper it would be sad if our testcases break one day because the old compat level is removed. Gbp-Dch: Ignore
2018-01-19allow the apt/lists/auxfiles/ directory to be missingDavid Kalnischkies
apt 1.6~alpha6 introduced aux requests to revamp the implementation of a-t-mirror. This already included the potential of running as non-root, but the detection wasn't complete resulting in errors or could produce spurious warnings along the way if the directory didn't exist yet. References: ef9677831f62a1554a888ebc7b162517d7881116 Closes: 887624
2018-01-03allow a method to request auxiliary filesDavid Kalnischkies
If a method needs a file to operate like e.g. mirror needs to get a list of mirrors before it can redirect the the actual requests to them. That could easily be solved by moving the logic into libapt directly, but by allowing a method to request other methods to do something we can keep this logic contained in the method and allow e.g. also methods which perform binary patching or similar things. Previously they would need to implement their own acquire system inside the existing one which in all likelyhood will not support the same features and methods nor operate with similar security compared to what we have already running 'above' the requesting method. That said, to avoid methods producing conflicts with "proper" files we are downloading a new directory is introduced to keep the auxiliary files in. [The message magic number 351 is a tribute to the german Grundgesetz article 35 paragraph 1 which defines that all authorities of the state(s) help each other on request.]
2017-12-13implement fallback to alternative URIs for all itemsDavid Kalnischkies
For deb files we always supported falling back from one server to the other if one failed to download the deb, but that was hardwired in the handling of this specific item. Moving this alongside the retry infrastructure we can implement it for all items and allow methods to use this as well by providing additional URIs in a redirect.
2017-12-13implement Acquire::Retries support for all itemsDavid Kalnischkies
Moving the Retry-implementation from individual items to the worker implementation not only gives every file retry capability instead of just a selected few but also avoids needing to implement it in each item (incorrectly).
2017-11-22tests: Improve handling profiling messages on CIJulian Andres Klode
We did not strip away profiling messages when we were diffing from stdin (-). Just always write temporary files and strip from them. We also had a problem when stripping ...profiling: from a line and the next line starts with profiling. Split the sed into two calls so we first remove complete profiling: lines before fixing the ...profiling: cases.
2017-10-05ignore unsupported key formats in apt-keyDavid Kalnischkies
gpg2 generates keyboxes by default and users end up putting either those or armored files into the trusted.gpg.d directory which apt tools neither expect nor can really work with without fortifying backward compatibility (at least under the ".gpg" extension). A (short) discussion about how to deal with keyboxes happened in https://lists.debian.org/deity/2017/07/msg00083.html As the last message in that thread is this changeset lets go ahead with it and see how it turns out. The idea is here simply that we check the first octal of a gpg file to have one of three accepted values. Testing on my machines has always produced just one of these, but running into those values on invalid files is reasonabily unlikely to not worry too much. Closes: #876508
2017-09-09test: Workaround gpgv warningJulian Andres Klode
gpgv: WARNING: This key is not suitable for signing in --compliance=gnupg mode
2017-07-26fail early in http if server answer is too small as wellDavid Kalnischkies
Failing on too much data is good, but we can do better by checking for exact filesizes as we know with hashsums how large a file should be, so if we get a file which has a size we do not expect we can drop it directly, regardless of if the file is larger or smaller than what we expect which should catch most cases which would end up as hashsum errors later now a lot sooner.
2017-06-27travis: ignore profiling warning in progress linesDavid Kalnischkies
On Travis CI running tests with code coverage enabled sometimes generates profiling lines, which we filter out for a while now, but that misses lines generated showing progress still causing test failures, so more sed logic is added in the hopes to ignore them. Extends: 58608941e6b58a46109b7cd875716b3d8054c4bf Gbp-Dch: Ignore
2017-06-26make the create-test-data script great againDavid Kalnischkies
Changes in the past to the buildsystem and the testing framework broke this little helper script – lets fix those problems to restore functionality. Gbp-Dch: Ignore
2017-06-26Call update from apt-key test for a strange path testDavid Kalnischkies
We setup a "horrible" environment in the apt-key testcase to check all kinds of things, but we really should be making also at least a simple apt update call, as that in turn will call apt-key which is how apt-key is used in the non-testcase world, so that calling should be able to deal with such environments as well. Gbp-Dch: Ignore
2017-06-26tests: fix gpg-agent killing in testcasesDavid Kalnischkies
We want to kill the agent if its home directory exists at that location, not if it isn't there (leaving an army of processes around). Gbp-Dch: Ignore
2017-01-19fix various typos reported by spellintianDavid Kalnischkies
Most of them in (old) code comments. The two instances of user visible string changes the po files of the manpages are fixed up as well. Gbp-Dch: Ignore Reported-By: spellintian
2016-12-31allow warning generation for non-whitelisted optionsDavid Kalnischkies
The idea is simple: Each¹ Find*( call starts with a call check if the given option (with the requested type) exists in the whitelist. The whitelist is specified via our configure-index file so that we have a better chance at keeping it current. the whitelist is loaded via a special (undocumented for now) configuration stanza and if none is loaded the empty whitelist will make it so that no warnings are shown. Much needs to be done still, but that is as good a time as any to take a snapshot of the current state and release it into the wild given that it found some bugs already and has no practical effect on users. ¹ not all in this iteration, but many
2016-12-21tests: cache the apt-key homedir used for Release signingDavid Kalnischkies
Importing a new secret key into gpg(2) can be increadibly slow which prolongs the test runs significantly – by caching the homedir we gain a significant speedbonus as reimporting already present keys seems like a far less costly operation. Git-Dch: Ignore
2016-11-25optional write aptwebserver log to client specific filesDavid Kalnischkies
The test test-handle-redirect-as-used-mirror-change serves multiple clients at the same time, so the order of the output is undefined and once in a while the two clients will intermix their lines causing the grep we perform on it later to fail making our tests fail. Solved by introducing client-specific logfiles which we all grep and sort the result to have the results more stable. Git-Dch: Ignore
2016-11-11apt-ftparchive: Support NotAutomatic and ButAutomaticUpgrades fieldsJames Clarke
This also changes Acquire-By-Hash to be "yes" rather than "true", so it is consistent with dak's output. Closes: #272557
2016-11-10travis: ignore profiling warning in test diffsDavid Kalnischkies
On Travis CI running tests with code coverage enabled sometimes generates lines like: profiling:/path/to/file.gcda:Merge mismatch for function 257 It would be nice if we could resolve this somehow as it garbles the statistics, but until then it is far more annoying that this causes test failures for no good reason. Gbp-Dch: Ignore
2016-11-09reset HOME, USER(NAME), TMPDIR & SHELL in DropPrivilegesDavid Kalnischkies
We can't cleanup the environment like e.g. sudo would do as you usually want the environment to "leak" into these helpers, but some variables like HOME should really not have still the value of the root user – it could confuse the helpers (USER) and HOME isn't accessible anyhow. Closes: 842877
2016-09-07test: Always install dpkg into our tests, regardless of MAJulian Andres Klode
Even if we only configure a single architecture, install dpkg, so dpkg can assert multi arch correctly. This also has the nice side effect of making single architecture and multiple architecture test cases more uniform. This fixes a regression from f878d3a862128bc1385616751ae1d78246b1bd01 ("test: Assert multi-arch in the chroot").
2016-09-07test: framework: Ensure copied status files have trailing linesJulian Andres Klode
If we copied one of the existing status files, we might not have a trailing newline, so let's add one. Gbp-Dch: ignore
2016-08-30test: Pass -d to dpkg-buildpackageJulian Andres Klode
This works around an issue on Fedora where dpkg complains about missing build-essential: dpkg-checkbuilddeps: Unmet build dependencies: build-essential:native Gbp-Dch: ignore
2016-08-29test, travis: Quieter testing with a new -qq modeJulian Andres Klode
Introduce a new -qq mode for our integration test framework, and make travis use it. The new -qq mode sets MSGLEVEL to 1. In MSGLEVEL=1, no messages are generated for passed tests, and all testcase filenames are printed in the same line. Also install first in travis, do not ls the installed output and run the install with chronic, so we only get output if it failed. Gbp-Dch: ignore
2016-08-29randomize acquire order for same type index filesDavid Kalnischkies
Without randomizing the order in which we download the index files we leak needlessly information to the mirrors of which architecture is native or foreign on this system. More importantly, we leak the order in which description translations will be used which in most cases will e.g. have the native tongue first. Note that the leak effect in practice is limited as apt detects if a file it wants to download is already available in the latest version from a previous download and does not query the server in such cases. Combined with the fact that Translation files are usually updated infrequently and not all at the same time, so a mirror can never be sure if it got asked about all files the user wants.
2016-08-27Merge branch 'portability/freebsd'Julian Andres Klode
2016-08-26test: Use :$(id -gn) instead of :root (when run as root)Julian Andres Klode
On BSD systems, the root group is wheel, not root, so let's just use the default group here. Gbp-Dch: ignore
2016-08-26test: Assert multi-arch in the chrootJulian Andres Klode
The host system might not have a dpkg installed, which makes dpkg fail with: dpkg not recorded as installed, cannot check for multi-arch support! That's entirely useless of course. We want to know if dpkg could support multi-arch in our chroot, so we pseudo-install dpkg into the chroot and pretend it's version is one version higher than the minimum dpkg version, so dpkg --assert-multi-arch works on recent dpkgs. Gbp-Dch: ignore
2016-08-26test: More portable check for dpkg versionsJulian Andres Klode
This check should work regardless if dpkg was installed by dpkg or by a native package manager like RPM or pkg. Gbp-Dch: ignore
2016-08-26test: Use a file to determine TEST_DEFAULT_GROUPJulian Andres Klode
This is more safe against sticky bits. For example, in FreeBSD all files created in /tmp have the group set to wheel. Gbp-Dch: ignore
2016-08-26test: Fix invalid quoting in variable expansionJulian Andres Klode
This breaks the tests with FreeBSD's shell, and is not needed - it works fine without it. Gbp-Dch: ignore
2016-08-26test: Use printf "%b\n" instead of echo for strings with '\'Julian Andres Klode
Use of echo with special characters is not portable. On a normal POSIX system, the behavior with backslash escaped strings is implementation-defined. On an XSI-conformant system, they must be interpreted. A way out is the printf command - printf "%b" specifies that the following argument is to be printed with backslash escapes interpreted. Gbp-Dch: ignore
2016-08-26test: Make sure we always find a dpkg in status fileJulian Andres Klode
Especially on non-Debian platforms, dpkg might not list itself on the host system, and thus dpkg --assert-multi-arch fails. Gbp-Dch: ignore
2016-08-26test: Explicitly pass --admindir=var/lib/dpkg to dpkgJulian Andres Klode
Our test suite assumes that dpkg's admindir is var/lib/dpkg. This might not always be true; for example, on FreeBSD, it is located at /var/db/dpkg. Gbp-Dch: ignore
2016-08-26test: Fix building of noopchrootJulian Andres Klode
Gbp-Dch: ignore
2016-08-26test: Get rid of debhelper rules.tiny example depJulian Andres Klode
Gbp-Dch: ignore
2016-08-26test: Allow stunnel to be used instead of stunnel4Julian Andres Klode
This is needed for Fedora and FreeBSD. Gbp-Dch: ignore
2016-08-26test: Substitute GNU commands for other commands where availableJulian Andres Klode
We are simply checking for gnuCMD and gCMD for each command we are interested in. Gbp-Dch: ignore
2016-08-26test: Use C locale instead of C.UTF-8Julian Andres Klode
Gbp-Dch: ignore
2016-08-26Make directory paths configurableJulian Andres Klode
This allows other vendors to use different paths, or to build your own APT in /opt for testing. Note that this uses + 1 in some places, as the paths we receive are absolute, but we need to strip of the initial /.
2016-08-24do not restore selections for already purged packagesDavid Kalnischkies
In most cases apt was already skipping the (re)setting of packages as to be removed/purged if dpkg had told us that it already did, but we haven't dealt with it in the most obvious of the cases: Selections set for packages we touched in this operation which either restores selections even dpkg would have overridden or e.g. tries to restore a purge selection for a package which was just purged – does not happen with apt itself as it isn't using selections in this way, but higher frontends like aptitude do. The result in the later case is a warning printed by dpkg that we try to set selections for an unknown package, which is harmless per se, but can be confusing for users and we really shouldn't cause warnings in dpkg if we can help it. Reported-By: Guillem Jover on IRC