fallback to well-known URI if by-hash fails

We uses a small trick to implement the fallback: We make it so, that
by-hash is a special compression algorithm and apt already knows how to
deal with fallback between compression algorithms.

The drawback with implementing this fallback is that a) we are guessing
again and more importantly b) by-hash is only tried for the first
compression algorithm we want to acquire, not for all as before – but
flipping between by-hash and well-known for each compression algorithm
seems to be not really worth it as it seems unlikely that there will
actually be mirrors who only mirror a subset of compressioned files, but
have by-hash enabled.

The user-experience is the usual fallback one: You see "Ign" lines in
the apt update output. The fallback is implemented as a transition
feature, so a (potentially huge) mirror network doesn't need a flagday.
It is not meant as a "someday we might" or "we don't, but some of our
mirrors might" option – we want to cut down on the 'Ign' lines front so
that they become meaningful – if we wanted to spam everyone with them, we
could enable by-hash by default for all repositories…
sources.list and config options are better suited for this.

Closes: 798919

1 files changed, 15 insertions, 11 deletions

diff --git a/test/integration/test-apt-by-hash-update b/test/integration/test-apt-by-hash-update
index c00ab497b..9b97bdeba 100755
--- a/test/integration/test-apt-by-hash-update
+++ b/test/integration/test-apt-by-hash-update
@@ -14,16 +14,15 @@ insertpackage 'unstable' 'foo' 'all' '1.0'
 setupaptarchive --no-update
 
 # make Packages *only* accessible by-hash for this test
-mkdir -p aptarchive/dists/unstable/main/binary-i386/by-hash/SHA512
-(cd  aptarchive/dists/unstable/main/binary-i386/by-hash/SHA512 && 
-     mv ../../Packages* . &&
-     ln -s Packages.gz  $(sha512sum Packages.gz|cut -f1 -d' ') )
-
-# add sources
-mkdir -p aptarchive/dists/unstable/main/source/by-hash/SHA512
-(cd  aptarchive/dists/unstable/main/source/by-hash/SHA512 && 
-     ln -s ../../Sources.gz  $(sha512sum ../../Sources.gz|cut -f1 -d' ') 
-)
+makebyhashonly() {
+	local NORMAL="$(readlink -f "./aptarchive/dists/unstable/main/${1}")"
+	local BYHASH="${NORMAL}/by-hash/SHA512"
+	mkdir -p "${BYHASH}"
+	find "${NORMAL}/" -maxdepth 1 -name "${2}*" -exec mv '{}' "$BYHASH" \;
+	ln -s "${BYHASH}/${2}.gz" "${BYHASH}/$(sha512sum "${BYHASH}/${2}.gz" | cut -f1 -d' ')"
+}
+makebyhashonly 'binary-i386' 'Packages'
+makebyhashonly 'source' 'Sources'
 
 ensureitsbroken() {
 	rm -rf rootdir/var/lib/apt/lists
@@ -39,7 +38,12 @@ ensureitsbroken -o Acquire::By-Hash=1
 
 ensureitworks() {
 	rm -rf rootdir/var/lib/apt/lists
-	testsuccess aptget update -o Acquire::Languages=none "$@"
+	testsuccess aptget update "$@" -o Acquire::Languages=none
+	testfailure grep '^Ign' rootdir/tmp/testsuccess.output
+	rm -rf rootdir/var/lib/apt/lists
+	testsuccess aptget update "$@"
+	cp -f rootdir/tmp/testsuccess.output rootdir/tmp/aptupdate.output
+	testsuccess grep '^Ign' rootdir/tmp/aptupdate.output
 	testsuccessequal "Inst foo (1.0 unstable [all])
 Conf foo (1.0 unstable [all])" aptget install -qq -s foo
 }