summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2016-01-14evaluate sourceslist-list-format entity in vendors sources.listDavid Kalnischkies
Parsing XML entity files in shell isn't exactly nice and causing the substitution with a while-read loop isn't either, but it seems to be good enough for the moment without changing too much. Thanks: Niels Thykier for reporting on IRC
2016-01-14change debian related entities to a more explicit nameDavid Kalnischkies
Git-Dch: Ignore
2016-01-14ensure built en-manpages are ignored by gitDavid Kalnischkies
Seems like the interaction between wildcards and exemptions has changed slightly, so be positively more explicit about our ignoring of en. Git-Dch: Ignore
2016-01-14mark not-declared helper function for showsrc as staticDavid Kalnischkies
Git-Dch: Ignore
2016-01-14Do not show multiple identical apt-cache showsrc entriesMichael Vogt
Closes: #734922
2016-01-12test-apt-acquire-additional-files: Set file mode of touched filesJulian Andres Klode
This prevents a test suite failure on systems with weird umasks. Also set umask 000 at the beginning so we can actually check for that anywhere. Gbp-Dch: ignore
2016-01-12Only enable pipelining if server is HTTP/1.1Julian Andres Klode
Just enabling it for anyone breaks with HTTP/1.0 servers and proxies sometimes. Closes: #810796
2016-01-12AUTHORS: Update: I am active, bubulle is notJulian Andres Klode
Gbp-Dch: ignore
2016-01-11Sort the list of sources to be built and linkedMattia Rizzolo
Fix reproducibility issue due to readdir() order by sorting the list of sources to be built and linked. [jak@debian.org: Added summary and fixed typo] Closes: #810509
2016-01-11search: Handle packages without descriptionJulian Andres Klode
If a package has no description, we would crash in search. While this should not happen, there seem to be some weird cases where it does. A safer way might be to make the whole parser thing safe against this, so pkgRecords::Lookup(Desc.FileList()) works and returns a parser where all values are empty. This would also fix all other instances of this bug, if there are any. Closes: #810622
2016-01-09Version the build-depends on liblz4-dev to >= 0.0~r126Julian Andres Klode
We need r126 of lz4, as this introduces the lz4frame.h header.
2016-01-09Bump dpkg-dev and debhelper build-dep to make lintian happyJulian Andres Klode
This is not really needed anymore, as those are in stable, but as they are versioned already, let's just do it. Gbp-Dch: ignore
2016-01-09Do not break apt-file (<< 3.0~exp1~), 2.X works fineJulian Andres Klode
There's no point in breaking all older apt-file versions just because one old experimental upload was broken.
2016-01-09NEWS: Refer to apt(8) instead of apt(1)Julian Andres Klode
Reported-By: Mattia Rizzolo (on IRC) Gbp-Dch: ignore
2016-01-08Release 1.2~exp11.2_exp1Julian Andres Klode
2016-01-08Break apt-file (<< 3.0~exp1~)Julian Andres Klode
Gbp-Dch: ignore
2016-01-08Install bash completion with correct nameJulian Andres Klode
Move the completion to completions/bash/apt and install all bash completions from completions/bash. Gbp-Dch: ignore
2016-01-08libapt-pkg5.0: Add Breaks appstream (<< 0.9.0-3~)Julian Andres Klode
This ensures that a compatible version of appstream is installed, that is, one that disables lz4 compression for its data.
2016-01-08Store the size of strings in the cacheJulian Andres Klode
By storing the size of the string in the cache, we can make use of it when comparing the names in the hashtable in pkgCache::FindGrp.
2016-01-08HashSumValue::Set: Do not provide const char* overloadJulian Andres Klode
Hide the std::string overload instead of providing a const char * one, the old variant was stupid. Gbp-Dch: ignore
2016-01-08debListParser: Convert another ParseDepends to StringViewJulian Andres Klode
I overlooked this Gbp-Dch: ignore
2016-01-08NEWS: Document recompression of indicesJulian Andres Klode
2016-01-08remove uncompressed leftover partial file before pdiff bootstrapDavid Kalnischkies
The code already deals with compressed leftovers, but forgot the uncompressed files. The opertunity is picked to reorder this code and add debug messages about the actions taken as well as produce such a leftover file in the associated testcase.
2016-01-08use filesize of compressed pdiffs for the limit if possibleDavid Kalnischkies
With the addition of the $HASH-Download field in the .diff/Index we got the size of the compressed patches for 'free', so if that information is available we can use it for a more fitting calculation of the size requirements of the patches vs. the complete file. Note that this predicts a too small size in the transition case in which the information isn't available for all patches, but figuring this out would be a lot of code for practically nothing as only one update can ever be in such a transition phase.
2016-01-08tests: limit autotest-functionname generation to sane charactersDavid Kalnischkies
Some (older) versions of bash seem to be allergic to a method named "aptautotest_grep_^apt" (note the caret). Unlikely that we are going to write autotests for such commands so we could just skip those, but lets instead just use "normal" characters in the names and strip the rest as we already did with the (arguable more common) '-'.
2016-01-08support '-' and no parameter for stdin in apt-helper cat-fileDavid Kalnischkies
This way it works more similar to the compressor binaries, which we can relief in this way from their job in the test framework avoiding the need of adding e.g. liblz4-tool to the test dependencies.
2016-01-08keep compressed indexes in a low-cost formatDavid Kalnischkies
Downloading and storing are two different operations were different compression types can be preferred. For downloading we provide the choice via Acquire::CompressionTypes::Order as there is a choice to be made between download size and speed – and limited by whats available in the repository. Storage on the other hand has all compressions currently supported by apt available and to reduce runtime of tools accessing these files the compression type should be a low-cost format in terms of decompression. apt traditionally stores its indexes uncompressed on disk, but has options to keep them compressed. Now that apt downloads additional files we also deal with files which simply can't be stored uncompressed as they are just too big (like Contents for apt-file). Traditionally they are downloaded in a low-cost format (gz) as repositories do not provide other formats, but there might be even lower-cost formats and for download we could introduce higher-cost in the repositories. Downloading an entire index potentially requires recompression to another format, so an update takes potentially longer – but big files are usually updated via pdiffs which has to de- and re-compress anyhow and does it on the fly anyhow, so there is no extra time needed and in general it seems to be benefitial to invest the time in update to save time later on file access.
2016-01-08tests: try to pick up compressors from config automaticallyDavid Kalnischkies
Less hardcoding should help while introducing new compressors. Git-Dch: Ignore
2016-01-08allow pdiff bootstrap from all supported compressorsDavid Kalnischkies
There is no reason to enforce that the file we start the bootstrap with is compressed with a compressor which is available online. This allows us to change the on-disk format as well as deals with repositories adding/removing support for a specific compressor.
2016-01-08ensure compression cleanup even without lists-cleanupDavid Kalnischkies
If we store files compressed in lists/ and the file switched compression formats we happened to retain the "old" format, but by default the cleanup process catched this oversight and removed the file. [The initial situation described doesn't arise as we store no files by default compressed and even with apt-file configuring Contents files, we don't really have that problem as there is just .gz files for those.] We solve this by just removing any uncompressed as well as compressed (we support) file just before we move the 'new' version of the file in.
2016-01-08use one 'store' method to rule all (de)compressorsDavid Kalnischkies
Adding a new compressor method meant adding a new method as well – even if that boilt down to just linking to our generalized decompressor with a new name. That is unneeded busywork if we can instead just call the generalized decompressor and let it figure out which compressor to use based on the filenames rather than by program name. For compatibility we ship still 'gzip', 'bzip2' and co, but they are just links to our "new" 'store' method.
2016-01-08include all compressed Packages/Sources files in Release fileDavid Kalnischkies
Having a hardcoded list of compression types here doesn't really provide us with anything beside added complexity each time someone adds a new compression type. That we don't need to be that specific is evident by Contents and Translation-* matchers which are a lot more generic and didn't generate problems anyhow.
2016-01-08fix typo in SrvRecords documentationDavid Kalnischkies
Git-Dch: Ignore
2016-01-08AvailableDescriptionLanguages: Use one string for all iterationsJulian Andres Klode
Do not create strings within the loop, that creates one string per language and does more work than needed. Instead, reserve enough space at the beginning and assign the prefix, and then resize and append inside the loop. Also call exists with the string itself instead of the c_str(), this means that the lookup uses the size information in the string now and does not have to call strlen() on it.
2016-01-08pkgCacheGenerator: CurMd5.Value() cannot be emptyJulian Andres Klode
It makes no sense to check if the value is empty, as it cannot be. It will always be a hexstring of exactly 32 bytes.
2016-01-08operator==(char*, StringView) use StringView.operator==Julian Andres Klode
Use the same path for both comparisons, as the operator== path is faster than just calling compare() - it avoids any comparison if the size differs. Gbp-Dch: ignore
2016-01-08pkgCacheGenerator::hash: Do not call tolower_ascii()Julian Andres Klode
Gbp-Dch: ignore
2016-01-08pkgCacheGenerator::StoreString: Get rid of std::stringJulian Andres Klode
Instead of storing a string -> map_stringitem_t mapping, create our own data type that can point to either a normal string or a string inside the cache. This avoids the creation of any string and improves performance slightly (about 4%).
2016-01-08Replace compare() == 0 checks with this == other checksJulian Andres Klode
This improves performance, as we now can ignore unequal strings based on their length already. Gbp-Dch: ignore
2016-01-08pkgCacheGenerator: Use StringView for toStringJulian Andres Klode
This removes some minor overhead. Gbp-Dch: ignore
2016-01-08pkgCacheGenerator::StoreString: Move the string into the mapJulian Andres Klode
Moving the string is likely faster than copying it. We could probably avoid strings alltogether in the future using some more crazy code, but I have not looked at that yet. Gbp-Dch: ignore
2016-01-07StringView: rfind: pos should be end of substr, not startJulian Andres Klode
Gbp-Dch: ignore
2016-01-07StringView: pos argument default should be nposJulian Andres Klode
Gbp-Dch: ignore
2016-01-07StringView::rfind(): Call rfind() instead of find() on substJulian Andres Klode
Thanks: Niels Thykier for reporting on IRC Gbp-Dch: ignore
2016-01-07Switch performance critical code to use APT::StringViewJulian Andres Klode
This improves performance of the cache generation on my ARM platform (4x Cortex A15) by about 10% to 20% from 2.35-2.50 to 2.1 seconds.
2016-01-07Introduce internal APT::StringView classJulian Andres Klode
The class APT::StringView implements a drop-in replacement for a subset of C++17 std::string_view() features. It will be dropped at a later point and may not be used in public interfaces.
2016-01-07rred: Run in parallelJulian Andres Klode
Remove the SingleInstance flag so we can use the new randomized queue feature to run parallel.
2016-01-07acquire: Allow parallelizing methods without hostsJulian Andres Klode
The maximum parallelization soft limit is the number of CPU cores * 2 on systems defining _SC_NPROCESSORS_ONLN. The hard limit in all cases is Acquire::QueueHost::Limit.
2016-01-07CopyFile: Use 64 * 1024 instead of 64000 as buffer sizeJulian Andres Klode
This is a multiple of the page size and thus results in less page faults, speeding up copying. Also, while we're at at, unify all uses of that size in a constant variable APT_BUFFER_SIZE.
2016-01-07apt-helper: cat-file: Add -C/--compress optionJulian Andres Klode
This allows passing compressing the output. The compressor must be a compressor name, extension, or an extension without the leading dot.