summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2016-11-14Use C locale instead of C.UTF-8 for protocol stringsJulian Andres Klode
The C.UTF-8 locale is not portable, so we need to use C, otherwise we crash on other systems. We can use std::locale::classic() for that, which might also be a bit cheaper than using locale("C"). (cherry picked from commit 0fb16c3e678044d6d06ba8a6199b1e96487ee0d8)
2016-11-14imbue .diff/Index parsing with C.UTF-8 as wellDavid Kalnischkies
In 3bdff17c894d0c3d0f813d358fc45d7a263f3552 we did it for the datetime parsing, but we use the same style in the parsing for pdiff (where the size of the file is in the middle of the three fields) so imbueing here as well is a good idea. (cherry picked from commit 1136a707b7792394ea4b1d039dda4f321fec9da4)
2016-11-14prevent C++ locale number formatting in text APIs (try 3)David Kalnischkies
This time it is the formatting of floating numbers in progress reporting with a radix charater potentially not being dot. Followup of 7303e11ff28f920a6277c159aa46f80c007350bb. Regression of b58e2c7c56b1416a343e81f9f80cb1f02c128e25 in so far as it exchanging very effected with slightly less effected code. LP: 1611010 (cherry picked from commit 0919f1df552ddf022ce4508cbf40e04eae5ef896)
2016-11-14prevent C++ locale number formatting in text APIs (try 2)David Kalnischkies
Followup of b58e2c7c56b1416a343e81f9f80cb1f02c128e25. Still a regression of sorts of 8b79c94af7f7cf2e5e5342294bc6e5a908cacabf. Closes: 832044 (cherry picked from commit 7303e11ff28f920a6277c159aa46f80c007350bb)
2016-11-14imbue datetime parsing with C.UTF-8 localeDavid Kalnischkies
Rewritten in 9febc2b238e1e322dce1f94ecbed46d595893b52 for c++ locales usage and rewritten again in 1d742e01470bba27715a8191c50adde4b39c2f19 to avoid a currently present stdlibc++6 bug in the std::get_time implementation. The later implementation uses still stringstreams for parsing, but forgot to explicitly reset the locale to something sane (for parsing english dates that is), so date and especially the parsing of a number is depending on the locale. Turns out, the French (among others) format their numbers with space as thousand separator so for some reason the stdlibc++6 thinks its a good idea to interpret the entire datetime string as a single number instead of realizing that in "25 Jun …" the later parts can't reasonably be part of that number even through there are spaces there… Workaround is hence: LC_NUMERIC=C.UTF-8 Closes: 828011 (cherry picked from commit 3bdff17c894d0c3d0f813d358fc45d7a263f3552)
2016-11-14avoid std::get_time usage to sidestep libstdc++6 bugDavid Kalnischkies
As reported upstream in https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71556 the implementation of std::get_time is currently not as accepting as strptime is, especially in how hours should be formatted. Just reverting 9febc2b238e1e322dce1f94ecbed46d595893b52 would be possible, but then we would reopen the problems fixed by it, so instead I opted here for a rewrite of the parsing logic which makes this method a lot longer, but at least it provides the same benefits as the rewrite in std::get_time was intended to give us and decouples us from the fix of the issue in the standard library implementation of GCC. LP: 1593583 (cherry picked from commit 1d742e01470bba27715a8191c50adde4b39c2f19)
2016-11-14accept only the expected UTC timezones in date parsingDavid Kalnischkies
HTTP/1.1 hardcodes GMT (RFC 7231 §7.1.1.1) and what is good enough for the internet must be good enough for us™ as we reuse the implementation internally to parse (most) dates we encounter in various places like the Release files with their Date and Valid-Until header fields. Implementing a fully timezone aware parser just feels too hard for no effective benefit as it would take 5+ years (= until LTS's are out of fashion) until a repository could use non-UTC dates and expect it to work. Not counting non-apt implementations which might or might not only want to encounter UTC here as well. As a bonus, this eliminates the use of an instance of setlocale in libapt. Closes: 819697 (cherry picked from commit 9febc2b238e1e322dce1f94ecbed46d595893b52)
2016-11-14use de-localed std::put_time instead rolling our ownDavid Kalnischkies
(cherry picked from commit eceb219c2a64f3f81421c3c6587380b6ae81a530)
2016-11-14avoid changing the global LC_TIME for Release writingDavid Kalnischkies
Using C++ here avoids calling setlocale here which never really was that ideal, but needed to avoid locale specific weekday/month names. (cherry picked from commit e0b01a85bd8395449a88e1806ea4a4e3acdbac33)
2016-10-31Release 1.2.151.2.15Julian Andres Klode
2016-10-051.2.15 RC1Julian Andres Klode
2016-10-05Do not read stderr from proxy autodetection scriptsJulian Andres Klode
This fixes a regression introduced in commit 8f858d560e3b7b475c623c4e242d1edce246025a don't leak FD in AutoProxyDetect command return parsing which accidentally made the proxy autodetection code also read the scripts output on stderr, not only on stdout when it switched the code from popen() to Popen(). Reported-By: Tim Small <tim@seoss.co.uk> (cherry picked from commit 0ecceb5bb9cc8727c117195945b7116aceb984fe)
2016-10-05VersionHash: Do not skip too long dependency linesJulian Andres Klode
If the dependency line does not contain spaces in the repository but does in the dpkg status file (because dpkg normalized the dependency list), the dpkg line might be longer than the line in the repository. If it now happens to be longer than 1024 characters, it would be skipped, causing the hashes to be out of date. Note that we have to bump the minor cache version again as this changes the format slightly, and we might get mismatches with an older src cache otherwise. Fixes Debian/apt#23 (cherry picked from commit 708e2f1fe99e6f067292bc909f03f12c181e4798)
2016-10-05abort connection on '.' target replies in SRVDavid Kalnischkies
Commit 3af3ac2f5ec007badeded46a94be2bd06b9917a2 (released in 1.3~pre1) implements proper fallback for SRV, but that works actually too good as the RFC defines that such an SRV record should indicate that the server doesn't provide this service and apt should respect this. The solution is hence to fail again as requested even if that isn't what the user (and perhaps even the server admins) wanted. At least we will print a message now explicitly mentioning SRV to point people in the right direction. Reported-In: https://bugs.kali.org/view.php?id=3525 Reported-By: Raphaël Hertzog (cherry picked from commit 99fdd8034b4a5cdb0100a33d0b3d5e26079c1695)
2016-10-05try not to call memcpy with length 0 in hash calculationsDavid Kalnischkies
memcpy is marked as nonnull for its input, but ignores the input anyhow if the declared length is zero. Our SHA2 implementations do this as well, it was "just" MD5 and SHA1 missing, so we add the length check here as well as along the callstack as it is really pointless to do all these method calls for "nothing". Reported-By: gcc -fsanitize=undefined (cherry picked from commit 644478e8db56f305601c3628a74e53de048b28c8)
2016-10-05Base256ToNum: Fix uninitialized valueJulian Andres Klode
If the inner Base256ToNum() returned false, it did not set Num to a new value, causing it to be uninitialized, and thus might have caused the function to exit despite a good result. Also document why the Res = Num, if (Res != Num) magic is done. Reported-By: valgrind (cherry picked from commit cf7503d8a09ebce695423fdeb2402c456c18f3d8)
2016-10-05TagFile: Fix off-by-one errors in comment strippingJulian Andres Klode
Adding 1 to the value of d->End - current makes restLength one byte too long: If we pass memchr(current, ..., restLength) has thus undefined behavior. Also, reading the value of current has undefined behavior if current >= d->End, not only for current > d->End: Consider a string of length 1, that is d->End = d->Current + 1. We can only read at d->Current + 0, but d->Current + 1 is beyond the end of the string. This probably caused several inexplicable build failures on hurd-i386 in the past, and just now caused a build failure on Ubuntu's amd64 builder. Reported-By: valgrind (cherry picked from commit 923c592ceb6014b31ec751b97b3ed659fa3e88ae)
2016-10-05Merge translations from 1.3~rc3Julian Andres Klode
This fixes a few fuzzy strings.
2016-10-05zh_CN.po: update simplified chinese translationZhou Mo
[jak@debian.org: This merges the state of 1.3_rc3]
2016-10-05test/integration/test-srcrecord: Make executableJulian Andres Klode
I actually tried to amend the previous commit, but apparently I forgot to add the file mode change. Gbp-Dch: ignore (cherry picked from commit 832f95f4d018f18ff7b3d0381206f25b5a4373a6)
2016-10-05Fix segfault and out-of-bounds read in Binary fieldsJulian Andres Klode
If a Binary field contains one or more spaces before a comma, the code produced a segmentation fault, as it accidentally set a pointer to 0 instead of the value of the pointer. If the comma is at the beginning of the field, the code would create a binStartNext that points one element before the start of the string, which is undefined behavior. We also need to check that we do not exit the string during the replacement of spaces before commas: A string of the form " ," would normally exit the boundary of the Buffer: binStartNext = offset 1 ',' binEnd = offset 0 ' ' isspace_ascii(*binEnd) = true => --binEnd => binEnd = - 1 We get rid of the problem by only allowing spaces to be eliminated if they are not the first character of the buffer: binStartNext = offset 1 ',' binEnd = offset 0 ' ' binEnd > buffer = false, isspace_ascii(*binEnd) = true => exit loop => binEnd remains 0 (cherry picked from commit ce6cd75dc367b92f65e4fb539dd166d0f3361f8c)
2016-10-05don't loop on pinning pkgs from absolute debs by regexDavid Kalnischkies
An absolute filename for a *.deb file starts with a /. A package with the name of the file is inserted in the cache which is provided by the "real" package for internal reasons. The pinning code detects a regex based wildcard by having the regex start with /. That is no problem as a / can not be included in a package name… expect that our virtual filename package can and does. We fix this two ways actually: First, a regex is only being considered a regex if it also ends with / (we don't support flags). That stops our problem with the virtual filename packages already, but to be sure we also do not enter the loop if matcher and package name are equal. It has to be noted that the creation of pins for virtual packages like the here effected filename packages is pointless as only versions can be pinned, but checking that a package is really purely virtual is too costly compared to just creating an unused pin. Closes: 835818 (cherry picked from commit e950b7e2f89b5e48192cd469c963a44fff9f1450)
2016-10-05changelog: Respect Dir setting for local changelog gettingJulian Andres Klode
This fixes issues with chroots, but the goal here was to get the test suite working on systems without dpkg. (cherry picked from commit 2ed62ba6abcad809d1898a40950f86217af73812)
2016-10-05apt-inst: debfile: Pass comp. Name to ExtractTar, not BinaryJulian Andres Klode
In the old days, apt-inst used to use binaries, but now it uses the built-in support and matches using Name, and not a Binary. (cherry picked from commit 8a362893a18eca569f8b93c572aaf966572b9546)
2016-10-05Accept --autoremove as alias for --auto-removeJulian Andres Klode
I probably missed that when I did the usability work. But better late than never. (cherry picked from commit 75d238ba66576c04f257e9d7c0a6995721f1441d)
2016-10-05install-progress: Call the real ::fork() in our fork() methodJulian Andres Klode
We basically called ourselves before, creating an endless loop. Reported-By: clang (cherry picked from commit d651c4cd71a43c385c3d3bcd3a9f25bf0a67f8f2)
2016-10-05Ignore SIGINT and SIGQUIT for Pre-Install hooksJulian Andres Klode
Instead of erroring out when receiving a SIGINT, let the child deal with it - we'll error out anyway if the child exits with an error or due to the signal. Also ignore SIGQUIT, as system() ignores it. This basically fixes Bug #832593, but: we are running the hooks via sh -c. Some shells exit with a signal error even if the command they are executing catches the signal and exits successfully. So far, this has been noticed on dash, which unfortunately, is our default shell. Example: $ cat trap.sh trap 'echo int' INT; sleep 10; exit 0 $ if dash -c ./trap.sh; then echo OK: $?; else echo FAIL: $?; fi ^Cint FAIL: 130 $ if mksh -c ./trap.sh; then echo OK: $?; else echo FAIL: $?; fi ^Cint OK: 0 $ if bash -c ./trap.sh; then echo OK: $?; else echo FAIL: $?; fi ^Cint OK: 0 (cherry picked from commit a6ae3d3df490e7a5a1c8324ba9dc2e63972b1529)
2016-10-05set the correct item FileSize in by-hash caseDavid Kalnischkies
In af81ab9030229b4ce6cbe28f0f0831d4896fda01 we implement by-hash as a special compression type, which breaks this filesize setting as the code is looking for a foobar.by-hash file then. Dealing this slightly gets us the intended value. Note that this has no direct effect as this value will be set in other ways, too, and could only effect progress reporting. Gbp-Dch: Ignore (cherry picked from commit 3084ef2292642d43e533654354a4929abe55d91b)
2016-10-05don't try pipelining if server closes connectionsDavid Kalnischkies
If a server closes a connection after sending us a file that tends to mean that its a type of server who always closes the connection – it is therefore relatively pointless to try pipelining with it even if it isn't a problem by itself: apt is just restarting the pipeline each time after it got served one file and the connection is closed. The problem starts if one or more proxies are between the server and apt and they disagree about how the connection should be as in the bugreporters case where the responses apt gets contain both Keep-Alive and Proxy-Connection headers (which apt both ignores) indicating a proxy is trying to keep a connection open while the response also contains "Connection: close" indicating the opposite which apt understands and respects as it is required to do. We avoid stepping into this abyss by not performing pipelining anymore if we got a respond with the indication to close connection if the response was otherwise a success – error messages are sent by some servers via this method as their pages tend to be created dynamically and hence their size isn't known a priori to them. Closes: #832113 (cherry picked from commit 9714d522056e5256f5a2de587d88eba7cb3291c2)
2016-10-05http(s): allow empty values for header fieldsDavid Kalnischkies
It seems completely pointless from a server-POV to sent empty header fields, so most of them don't do it (simply proven by this limitation existing since day one) – but it is technically allowed by the RFC as the surounding whitespaces are optional and Github seems to like sending "X-Geo-Block-List:\r\n" since recently (bug reports in other http clients indicate July) at least sometimes as the reporter claims to have seen it on https only even through it can happen with both. Closes: 834048 (cherry picked from commit 148c049150cc39f2e40894c1684dc2aefea1117e)
2016-10-05drop incorrect const attribute from DirectoryExistsDavid Kalnischkies
Since its existence in 2010 DirectoryExists was always marked with this attribute, but for no real reason. Arguably a check for the existence of the file is not modifying global state, so theoretically this shouldn't be a problem. It is wrong from a logical point of view through as between two calls the directory could be created so the promise we made to the compiler that it could remove the second call would be wrong, so API wise it is wrong. It's a bit mysterious that this is only observeable on ppc64el and can be fixed by reordering code ever so slightly, but in the end its more our fault for adding this attribute than the compilers fault for doing something silly based on the attribute. LP: 1473674 (cherry picked from commit 9445fa62386c80c9822e77484d30b2109aa0f2dc)
2016-10-05fileutl: empty file support: Avoid fstat() on -1 fd and check resultJulian Andres Klode
When checking if a file is empty, we forget to check that fstat() actually worked. (cherry picked from commit 15fe8e62d37bc87114c59d385bed7ceefb72886b)
2016-10-05allow user@host (aka: no password) in URI parsingDavid Kalnischkies
If the URI had no password the username was ignored (cherry picked from commit a1f3ac8aba0675321dd46d074af8abcbb10c19fd)
2016-10-05pass --force-remove-essential to dpkg only if neededDavid Kalnischkies
APT (usually) knows which package is essential or not, so we can avoid passing this force flag to dpkg unconditionally if the user hasn't chosen a non-default essential handling obscuring the information. (cherry picked from commit d3930f8716f439c229cd3d11813823d847a2ecff)
2016-10-05gpgv: Unlink the correct temp file in error caseJulian Andres Klode
Previously, when data could be created and sig not, we would unlink sig, not data (and vice versa). (cherry picked from commit d0d06f44ed60a3888528d834a799bae86c2978d5)
2016-10-05if the FileFd failed already following calls should fail, tooDavid Kalnischkies
There is no point in trying to perform Write/Read on a FileFd which already failed as they aren't going to work as expected, so we should make sure that they fail early on and hard. (cherry picked from commit 02c38073af51802c02bb104d4450e0e112d641ad)
2016-10-05(error) va_list 'args' was opened but not closed by va_end()David Kalnischkies
Reported-By: cppcheck Gbp-Dch: Ignore (cherry picked from commit 196d590a99e309764e07c9dc23ea98897eebf53a)
2016-08-31rred: truncate result file before writing to itDavid Kalnischkies
If another file in the transaction fails and hence dooms the transaction we can end in a situation in which a -patched file (= rred writes the result of the patching to it) remains in the partial/ directory. The next apt call will perform the rred patching again and write its result again to the -patched file, but instead of starting with an empty file as intended it will override the content previously in the file which has the same result if the new content happens to be longer than the old content, but if it isn't parts of the old content remain in the file which will pass verification as the new content written to it matches the hashes and if the entire transaction passes the file will be moved the lists/ directory where it might or might not trigger errors depending on if the old content which remained forms a valid file together with the new content. This has no real security implications as no untrusted data is involved: The old content consists of a base file which passed verification and a bunch of patches which all passed multiple verifications as well, so the old content isn't controllable by an attacker and the new one isn't either (as the new content alone passes verification). So the best an attacker can do is letting the user run into the same issue as in the report. Closes: #831762 (cherry picked from commit 0e071dfe205ad21d8b929b4bb8164b008dc7c474)
2016-08-31use proper warning for automatic pipeline disableDavid Kalnischkies
Also fixes message itself to mention the correct option name as noticed in #832113. (cherry picked from commit b9c20219dc17db1d29eaf297263a4b008bd1b90b)
2016-08-31verify hash of input file in rredDavid Kalnischkies
We read the entire input file we want to patch anyhow, so we can also calculate the hash for that file and compare it with what he had expected it to be. Note that this isn't really a security improvement as a) the file we patch is trusted & b) if the input is incorrect, the result will hardly be matching, so this is just for failing slightly earlier with a more relevant error message (althrough, in terms of rred its ignored and complete download attempt instead). (cherry picked from commit 6e71ec6fcdcaa926c98fa58cd4af38e42556df15)
2016-08-31call flush on the wrapped writebuffered FileFdDavid Kalnischkies
The flush call is a no-op in most FileFd implementations so this isn't as critical as it might sound as the only non-trivial implementation is in the buffered writer, which tends not be used to buffer another buffer… (cherry picked from commit 8ca481e8419c19b6ef9074b68cc028177a507161)
2016-08-31Turkish program translation updateMert Dirik
Closes: 832039 (cherry picked from commit a913e64ead6ada2adae6fb5f35212187ad5acd01)
2016-08-31Add kernels with "+" in the package name to APT::NeverAutoRemoveAndrew Patterson
Escape "+" in kernel package names when generating APT::NeverAutoRemove list so it is not treated as a regular expression meta-character. [Changed by David Kalnischkies: let test actually test the change] Closes: #830159 (cherry picked from commit 130176bcb6ce65c98d5692196c55cc18b4c210e0)
2016-08-31keep trying with next if connection to a SRV host failedDavid Kalnischkies
Instead of only trying the first host we get via SRV, we try them all as we are supposed to and if that isn't working we try to connect to the host itself as if we hadn't seen any SRV records. This was already the intend of the old code, but it failed to hide earlier problems for the next call, which would unconditionally fail then resulting in an all around failure to connect. With proper stacking we can also keep the error messages of each call around (and in the order tried) so if the entire connection fails we can report all the things we have tried while we discard the entire stack if something works out in the end. (cherry picked from commit 3af3ac2f5ec007badeded46a94be2bd06b9917a2)
2016-08-31report all instead of first error up the acquire chainDavid Kalnischkies
If we don't give a specific error to report up it is likely that all error currently in the error stack are equally important, so reporting just one could turn out to be confusing e.g. if name resolution failed in a SRV record list. (cherry picked from commit b50dfa6b2dd2d459e0c2746ac9367982b96ffac0)
2016-08-31don't change owner/perms/times through file:// symlinksDavid Kalnischkies
If we have files in partial/ from a previous invocation or similar such those could be symlinks created by file:// sources. The code is expecting only real files through and happily changes owner, modification times and permission on the file the symlink points to which tend to be files we have no business in touching in this way. Permissions of symlinks shouldn't be changed, changing owner is usually pointless to, but just to be sure we pick the easy way out and use lchown, check for symlinks before chmod/utimes. Reported-By: Mattia Rizzolo on IRC (cherry picked from commit 3465138575e1fd0d5892d9b6be1ae232eb873460)
2016-08-31use the right key for compressor configuration dumpDavid Kalnischkies
The generated dump output is incorrect in sofar as it uses the name as the key for this compressor, but they don't need to be equal as is the case if you force some of the inbuilt ones to be disabled as our testing framework does it at times. This is hidden from changelog as nobody will actually notice while describing it in a few words make it sound like an important change… Git-Dch: Ignore (cherry picked from commit 52bafeade99b700eeb4585608c5eee086b94dfa8)
2016-08-31avoid 416 response teardown binding to null pointerDavid Kalnischkies
methods/http.cc:640:13: runtime error: reference binding to null pointer of type 'struct FileFd' This reference is never used in the cases it has a nullptr, so the practical difference is non-existent, but its a bug still. Reported-By: gcc -fsanitize=undefined (cherry picked from commit 4460551841d909d3ee9c1de00156ed3cdf8b1665)
2016-08-31Make the test case executableJulian Andres Klode
Gbp-Dch: ignore (cherry picked from commit 2a90aa7a064047fb1c8783b31720cd345018ca4a)
2016-08-31indextargets: Check that cache could be built before using itJulian Andres Klode
This caused a crash because the cache was a nullptr. Closes: #829651 (cherry picked from commit 8823972649b0d3049c9c0d34b5f1d31160234fb4)