summaryrefslogtreecommitdiff
path: root/apt-pkg/contrib
AgeCommit message (Collapse)Author
2016-11-14Use C locale instead of C.UTF-8 for protocol stringsJulian Andres Klode
The C.UTF-8 locale is not portable, so we need to use C, otherwise we crash on other systems. We can use std::locale::classic() for that, which might also be a bit cheaper than using locale("C"). (cherry picked from commit 0fb16c3e678044d6d06ba8a6199b1e96487ee0d8)
2016-11-14imbue datetime parsing with C.UTF-8 localeDavid Kalnischkies
Rewritten in 9febc2b238e1e322dce1f94ecbed46d595893b52 for c++ locales usage and rewritten again in 1d742e01470bba27715a8191c50adde4b39c2f19 to avoid a currently present stdlibc++6 bug in the std::get_time implementation. The later implementation uses still stringstreams for parsing, but forgot to explicitly reset the locale to something sane (for parsing english dates that is), so date and especially the parsing of a number is depending on the locale. Turns out, the French (among others) format their numbers with space as thousand separator so for some reason the stdlibc++6 thinks its a good idea to interpret the entire datetime string as a single number instead of realizing that in "25 Jun …" the later parts can't reasonably be part of that number even through there are spaces there… Workaround is hence: LC_NUMERIC=C.UTF-8 Closes: 828011 (cherry picked from commit 3bdff17c894d0c3d0f813d358fc45d7a263f3552)
2016-11-14avoid std::get_time usage to sidestep libstdc++6 bugDavid Kalnischkies
As reported upstream in https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71556 the implementation of std::get_time is currently not as accepting as strptime is, especially in how hours should be formatted. Just reverting 9febc2b238e1e322dce1f94ecbed46d595893b52 would be possible, but then we would reopen the problems fixed by it, so instead I opted here for a rewrite of the parsing logic which makes this method a lot longer, but at least it provides the same benefits as the rewrite in std::get_time was intended to give us and decouples us from the fix of the issue in the standard library implementation of GCC. LP: 1593583 (cherry picked from commit 1d742e01470bba27715a8191c50adde4b39c2f19)
2016-11-14accept only the expected UTC timezones in date parsingDavid Kalnischkies
HTTP/1.1 hardcodes GMT (RFC 7231 §7.1.1.1) and what is good enough for the internet must be good enough for us™ as we reuse the implementation internally to parse (most) dates we encounter in various places like the Release files with their Date and Valid-Until header fields. Implementing a fully timezone aware parser just feels too hard for no effective benefit as it would take 5+ years (= until LTS's are out of fashion) until a repository could use non-UTC dates and expect it to work. Not counting non-apt implementations which might or might not only want to encounter UTC here as well. As a bonus, this eliminates the use of an instance of setlocale in libapt. Closes: 819697 (cherry picked from commit 9febc2b238e1e322dce1f94ecbed46d595893b52)
2016-11-14use de-localed std::put_time instead rolling our ownDavid Kalnischkies
(cherry picked from commit eceb219c2a64f3f81421c3c6587380b6ae81a530)
2016-10-05Do not read stderr from proxy autodetection scriptsJulian Andres Klode
This fixes a regression introduced in commit 8f858d560e3b7b475c623c4e242d1edce246025a don't leak FD in AutoProxyDetect command return parsing which accidentally made the proxy autodetection code also read the scripts output on stderr, not only on stdout when it switched the code from popen() to Popen(). Reported-By: Tim Small <tim@seoss.co.uk> (cherry picked from commit 0ecceb5bb9cc8727c117195945b7116aceb984fe)
2016-10-05try not to call memcpy with length 0 in hash calculationsDavid Kalnischkies
memcpy is marked as nonnull for its input, but ignores the input anyhow if the declared length is zero. Our SHA2 implementations do this as well, it was "just" MD5 and SHA1 missing, so we add the length check here as well as along the callstack as it is really pointless to do all these method calls for "nothing". Reported-By: gcc -fsanitize=undefined (cherry picked from commit 644478e8db56f305601c3628a74e53de048b28c8)
2016-10-05Base256ToNum: Fix uninitialized valueJulian Andres Klode
If the inner Base256ToNum() returned false, it did not set Num to a new value, causing it to be uninitialized, and thus might have caused the function to exit despite a good result. Also document why the Res = Num, if (Res != Num) magic is done. Reported-By: valgrind (cherry picked from commit cf7503d8a09ebce695423fdeb2402c456c18f3d8)
2016-10-05drop incorrect const attribute from DirectoryExistsDavid Kalnischkies
Since its existence in 2010 DirectoryExists was always marked with this attribute, but for no real reason. Arguably a check for the existence of the file is not modifying global state, so theoretically this shouldn't be a problem. It is wrong from a logical point of view through as between two calls the directory could be created so the promise we made to the compiler that it could remove the second call would be wrong, so API wise it is wrong. It's a bit mysterious that this is only observeable on ppc64el and can be fixed by reordering code ever so slightly, but in the end its more our fault for adding this attribute than the compilers fault for doing something silly based on the attribute. LP: 1473674 (cherry picked from commit 9445fa62386c80c9822e77484d30b2109aa0f2dc)
2016-10-05fileutl: empty file support: Avoid fstat() on -1 fd and check resultJulian Andres Klode
When checking if a file is empty, we forget to check that fstat() actually worked. (cherry picked from commit 15fe8e62d37bc87114c59d385bed7ceefb72886b)
2016-10-05allow user@host (aka: no password) in URI parsingDavid Kalnischkies
If the URI had no password the username was ignored (cherry picked from commit a1f3ac8aba0675321dd46d074af8abcbb10c19fd)
2016-10-05gpgv: Unlink the correct temp file in error caseJulian Andres Klode
Previously, when data could be created and sig not, we would unlink sig, not data (and vice versa). (cherry picked from commit d0d06f44ed60a3888528d834a799bae86c2978d5)
2016-10-05if the FileFd failed already following calls should fail, tooDavid Kalnischkies
There is no point in trying to perform Write/Read on a FileFd which already failed as they aren't going to work as expected, so we should make sure that they fail early on and hard. (cherry picked from commit 02c38073af51802c02bb104d4450e0e112d641ad)
2016-10-05(error) va_list 'args' was opened but not closed by va_end()David Kalnischkies
Reported-By: cppcheck Gbp-Dch: Ignore (cherry picked from commit 196d590a99e309764e07c9dc23ea98897eebf53a)
2016-08-31call flush on the wrapped writebuffered FileFdDavid Kalnischkies
The flush call is a no-op in most FileFd implementations so this isn't as critical as it might sound as the only non-trivial implementation is in the buffered writer, which tends not be used to buffer another buffer… (cherry picked from commit 8ca481e8419c19b6ef9074b68cc028177a507161)
2016-08-31don't change owner/perms/times through file:// symlinksDavid Kalnischkies
If we have files in partial/ from a previous invocation or similar such those could be symlinks created by file:// sources. The code is expecting only real files through and happily changes owner, modification times and permission on the file the symlink points to which tend to be files we have no business in touching in this way. Permissions of symlinks shouldn't be changed, changing owner is usually pointless to, but just to be sure we pick the easy way out and use lchown, check for symlinks before chmod/utimes. Reported-By: Mattia Rizzolo on IRC (cherry picked from commit 3465138575e1fd0d5892d9b6be1ae232eb873460)
2016-08-31don't do atomic overrides with failed filesDavid Kalnischkies
We deploy atomic renames for some files, but these renames also happen if something about the file failed which isn't really the point of the exercise… Closes: 828908 (cherry picked from commit fc5db01bb7d1546944200d197866b0b5c378f100)
2016-06-20do not error if auto-detect-proxy cmd has no outputDavid Kalnischkies
Regression introduced in 8f858d560e3b7b475c623c4e242d1edce246025a. Commands are probably better of always having output through as the fall through to the generic proxy settings is likely not intended. As documenting and implementing this more consistently is kind of a regression through, it is split off into the next commit. Closes: 827713 (cherry picked from commit cad1877559f3e1703c3fea4d081978e1b4bb4a0e)
2016-06-10don't leak FD in AutoProxyDetect command return parsingDavid Kalnischkies
Just closing the fd would be enough, but while we are at it we can also use the Popen interface to have an easier time with this. (cherry picked from commit 8f858d560e3b7b475c623c4e242d1edce246025a)
2016-06-10don't leak an FD in lz4 (de)compressionDavid Kalnischkies
Seen first in #826783, but as this buglog also shows leaked uncompressed files as well we don't close it just yet. (cherry picked from commit 6f35be91c9e86e463bca7df6eadf05412c7b732c)
2016-06-10do not hang on piped input in PipedFileFdPrivateDavid Kalnischkies
This effects only compressors configured on the fly (rather then the inbuilt ones as they use a library). (cherry picked from commit bdc42211700ef0f6f40e4ef3f362e52d684d70fb)
2016-06-01prevent C++ locale number formatting in text APIsDavid Kalnischkies
Setting the C++ locale via std::locale::global(std::locale("")); which would otherwise default to the default C locale (aka: unaffected by setlocale) effects the formatting of numeric types in IO streams, which for output for humans is perfectly sensible, but breaks our many text interfaces used and parsed by us and others without expecting the numbers to be formatted. Closes: #825396 (cherry picked from commit b58e2c7c56b1416a343e81f9f80cb1f02c128e25)
2016-04-03use buffered writing for InRelease splittingDavid Kalnischkies
Hardly noticeable, but given that we have the option to easily enable it, lets enable it as every newline in the message is written individually by the code.
2016-04-03consistently add APT_OVERRIDE as method markerDavid Kalnischkies
Some methods had it missing, some used the keyword directly, which isn't a problem as it is a cc file, but for consistency lets stick to our macro for now. Git-Dch: Ignore
2016-03-28Allow lowering trust level of a hash via configJulian Andres Klode
Introduces APT::Hashes::<NAME> with entries Untrusted and Weak which can be set to true to cause the hash to be treated as untrusted and/or weak.
2016-03-24Set hidden visibility for StringViewJulian Andres Klode
This avoids templates using StringView to be exported, such as std::vector<StringView*>::emplace_back(). Gbp-Dch: ignore
2016-03-13Do not consider SHA1 usableJulian Andres Klode
SHA1 is not reasonably secure anymore, so we should not consider it usable anymore. The test suite is adjusted to account for this.
2016-03-07Fix several typosVeres Lajos
This effectively merges branch 'typofixes-vlajos-20150807' of github.com:vlajos/apt with the following commit: commit 13cacb3e2e2352ba701e769fc889e3344fabbf7e Author: Veres Lajos <vlajos@gmail.com> Date: Sun Aug 9 00:12:53 2015 +0100 typofix - https://github.com/vlajos/misspell_fixer It has been rebased for a better commit message.
2016-03-06Fix lzma write support to handle "try again" caseColin Watson
The liblzma-based write code needs the same tweaks that the read code already has to cope with the situation where lzma_code returns zero the first time through because avail_out is zero, but will do more work if called again. This ports the read tweaks to the write code as closely as possible (including matching comments etc.). Closes: #751688
2016-02-10Set seekpos to 0 after reopening files in Seek()Julian Andres Klode
If we just reopened the file, we also need to reset the current seek position when we reset the buffer, otherwise the code will not try to seek to the position given to Skip (from 0), but will try to seek to old offset + the position given to skip. Closes: #812994, #813000
2016-02-04Use some semantically more correct buffer operationsJulian Andres Klode
When writing into the buffer write to free() bytes starting at getend(), instead of buffersize_max bytes at get() -> get() is a read pointer. This makes no difference in practice though, as we reset the buffer before the call, so start = end = 0. Gbp-Dch: ignore
2016-02-04Correctly report write errors when flushing buffered writerJulian Andres Klode
We cannot just return false without setting an error, as InternalWrite does not set one itself.
2016-02-03Avoid temporary strings in SubstVar.Adrian Wielgosik
Microoptimization, but still gives a measurable 2-3% improvement when using commands with lots of output like `apt list`.
2016-02-01Do not buffer writes larger than the buffer if possibleJulian Andres Klode
It makes no sense to split a large block into multiple small blocks, so when we have the chance to write them unbuffered, do so.
2016-02-01BufferedWriteFileFdPrivate: Simplify InternalWrite()Julian Andres Klode
We do not need the loop, FileFd::Private() handles this for us. Gbp-Dch: ignore
2016-02-01simple_buffer::write: Use free() instead of maxsize - size()Julian Andres Klode
We want to check whether the amount of free space is smaller than the requested write size. Checking maxsize - size() is incorrect for bufferstart >= 0, as size() = end - start. Gbp-Dch: ignore
2016-01-30fix "Mismatched free() / delete / delete []" in simple_bufferStefan Bühler
2016-01-26drop explicit check for EWOULDBLOCK if it has the same value as EAGAINDavid Kalnischkies
gcc correctly reports that we check for the same value twice, expect that the manpage of read(2) tells us to do it for portability, so to make both sides happy lets add a little #if'ing here. Reported-By: gcc-6 Git-Dch: Ignore
2016-01-15string_view: Drop constexpr constructor for standard compatibilityJulian Andres Klode
APT::StringView is supposed to be a temporary measure, until support for the standardized string_view is widely available. Introducing additional unstandardized features just makes porting to the standard version harder. The constexpr constructor also won't have any real effect on most systems, as the compiler will happily optimise the strlen() call away for constant strings. Gbp-Dch: ignore
2016-01-15provide a constexpr char[] overload for APT::StringViewDavid Kalnischkies
The commit also adds a few trivial tests Git-Dch: Ignore
2016-01-15return correct position in APT::StringView::(r)findDavid Kalnischkies
The position returned is supposed to be the position of the character counted from the start of the string, but if we used the substr calling overloads the skipped over prefix wasn't considered. The pos parameter of rfind had also the wrong semantic.
2016-01-08Store the size of strings in the cacheJulian Andres Klode
By storing the size of the string in the cache, we can make use of it when comparing the names in the hashtable in pkgCache::FindGrp.
2016-01-08HashSumValue::Set: Do not provide const char* overloadJulian Andres Klode
Hide the std::string overload instead of providing a const char * one, the old variant was stupid. Gbp-Dch: ignore
2016-01-08operator==(char*, StringView) use StringView.operator==Julian Andres Klode
Use the same path for both comparisons, as the operator== path is faster than just calling compare() - it avoids any comparison if the size differs. Gbp-Dch: ignore
2016-01-07StringView: rfind: pos should be end of substr, not startJulian Andres Klode
Gbp-Dch: ignore
2016-01-07StringView: pos argument default should be nposJulian Andres Klode
Gbp-Dch: ignore
2016-01-07StringView::rfind(): Call rfind() instead of find() on substJulian Andres Klode
Thanks: Niels Thykier for reporting on IRC Gbp-Dch: ignore
2016-01-07Switch performance critical code to use APT::StringViewJulian Andres Klode
This improves performance of the cache generation on my ARM platform (4x Cortex A15) by about 10% to 20% from 2.35-2.50 to 2.1 seconds.
2016-01-07Introduce internal APT::StringView classJulian Andres Klode
The class APT::StringView implements a drop-in replacement for a subset of C++17 std::string_view() features. It will be dropped at a later point and may not be used in public interfaces.
2016-01-07CopyFile: Use 64 * 1024 instead of 64000 as buffer sizeJulian Andres Klode
This is a multiple of the page size and thus results in less page faults, speeding up copying. Also, while we're at at, unify all uses of that size in a constant variable APT_BUFFER_SIZE.