summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2016-01-09NEWS: Refer to apt(8) instead of apt(1)Julian Andres Klode
Reported-By: Mattia Rizzolo (on IRC) Gbp-Dch: ignore
2016-01-08Release 1.2~exp11.2_exp1Julian Andres Klode
2016-01-08Break apt-file (<< 3.0~exp1~)Julian Andres Klode
Gbp-Dch: ignore
2016-01-08Install bash completion with correct nameJulian Andres Klode
Move the completion to completions/bash/apt and install all bash completions from completions/bash. Gbp-Dch: ignore
2016-01-08libapt-pkg5.0: Add Breaks appstream (<< 0.9.0-3~)Julian Andres Klode
This ensures that a compatible version of appstream is installed, that is, one that disables lz4 compression for its data.
2016-01-08Store the size of strings in the cacheJulian Andres Klode
By storing the size of the string in the cache, we can make use of it when comparing the names in the hashtable in pkgCache::FindGrp.
2016-01-08HashSumValue::Set: Do not provide const char* overloadJulian Andres Klode
Hide the std::string overload instead of providing a const char * one, the old variant was stupid. Gbp-Dch: ignore
2016-01-08debListParser: Convert another ParseDepends to StringViewJulian Andres Klode
I overlooked this Gbp-Dch: ignore
2016-01-08NEWS: Document recompression of indicesJulian Andres Klode
2016-01-08remove uncompressed leftover partial file before pdiff bootstrapDavid Kalnischkies
The code already deals with compressed leftovers, but forgot the uncompressed files. The opertunity is picked to reorder this code and add debug messages about the actions taken as well as produce such a leftover file in the associated testcase.
2016-01-08use filesize of compressed pdiffs for the limit if possibleDavid Kalnischkies
With the addition of the $HASH-Download field in the .diff/Index we got the size of the compressed patches for 'free', so if that information is available we can use it for a more fitting calculation of the size requirements of the patches vs. the complete file. Note that this predicts a too small size in the transition case in which the information isn't available for all patches, but figuring this out would be a lot of code for practically nothing as only one update can ever be in such a transition phase.
2016-01-08tests: limit autotest-functionname generation to sane charactersDavid Kalnischkies
Some (older) versions of bash seem to be allergic to a method named "aptautotest_grep_^apt" (note the caret). Unlikely that we are going to write autotests for such commands so we could just skip those, but lets instead just use "normal" characters in the names and strip the rest as we already did with the (arguable more common) '-'.
2016-01-08support '-' and no parameter for stdin in apt-helper cat-fileDavid Kalnischkies
This way it works more similar to the compressor binaries, which we can relief in this way from their job in the test framework avoiding the need of adding e.g. liblz4-tool to the test dependencies.
2016-01-08keep compressed indexes in a low-cost formatDavid Kalnischkies
Downloading and storing are two different operations were different compression types can be preferred. For downloading we provide the choice via Acquire::CompressionTypes::Order as there is a choice to be made between download size and speed – and limited by whats available in the repository. Storage on the other hand has all compressions currently supported by apt available and to reduce runtime of tools accessing these files the compression type should be a low-cost format in terms of decompression. apt traditionally stores its indexes uncompressed on disk, but has options to keep them compressed. Now that apt downloads additional files we also deal with files which simply can't be stored uncompressed as they are just too big (like Contents for apt-file). Traditionally they are downloaded in a low-cost format (gz) as repositories do not provide other formats, but there might be even lower-cost formats and for download we could introduce higher-cost in the repositories. Downloading an entire index potentially requires recompression to another format, so an update takes potentially longer – but big files are usually updated via pdiffs which has to de- and re-compress anyhow and does it on the fly anyhow, so there is no extra time needed and in general it seems to be benefitial to invest the time in update to save time later on file access.
2016-01-08tests: try to pick up compressors from config automaticallyDavid Kalnischkies
Less hardcoding should help while introducing new compressors. Git-Dch: Ignore
2016-01-08allow pdiff bootstrap from all supported compressorsDavid Kalnischkies
There is no reason to enforce that the file we start the bootstrap with is compressed with a compressor which is available online. This allows us to change the on-disk format as well as deals with repositories adding/removing support for a specific compressor.
2016-01-08ensure compression cleanup even without lists-cleanupDavid Kalnischkies
If we store files compressed in lists/ and the file switched compression formats we happened to retain the "old" format, but by default the cleanup process catched this oversight and removed the file. [The initial situation described doesn't arise as we store no files by default compressed and even with apt-file configuring Contents files, we don't really have that problem as there is just .gz files for those.] We solve this by just removing any uncompressed as well as compressed (we support) file just before we move the 'new' version of the file in.
2016-01-08use one 'store' method to rule all (de)compressorsDavid Kalnischkies
Adding a new compressor method meant adding a new method as well – even if that boilt down to just linking to our generalized decompressor with a new name. That is unneeded busywork if we can instead just call the generalized decompressor and let it figure out which compressor to use based on the filenames rather than by program name. For compatibility we ship still 'gzip', 'bzip2' and co, but they are just links to our "new" 'store' method.
2016-01-08include all compressed Packages/Sources files in Release fileDavid Kalnischkies
Having a hardcoded list of compression types here doesn't really provide us with anything beside added complexity each time someone adds a new compression type. That we don't need to be that specific is evident by Contents and Translation-* matchers which are a lot more generic and didn't generate problems anyhow.
2016-01-08fix typo in SrvRecords documentationDavid Kalnischkies
Git-Dch: Ignore
2016-01-08AvailableDescriptionLanguages: Use one string for all iterationsJulian Andres Klode
Do not create strings within the loop, that creates one string per language and does more work than needed. Instead, reserve enough space at the beginning and assign the prefix, and then resize and append inside the loop. Also call exists with the string itself instead of the c_str(), this means that the lookup uses the size information in the string now and does not have to call strlen() on it.
2016-01-08pkgCacheGenerator: CurMd5.Value() cannot be emptyJulian Andres Klode
It makes no sense to check if the value is empty, as it cannot be. It will always be a hexstring of exactly 32 bytes.
2016-01-08operator==(char*, StringView) use StringView.operator==Julian Andres Klode
Use the same path for both comparisons, as the operator== path is faster than just calling compare() - it avoids any comparison if the size differs. Gbp-Dch: ignore
2016-01-08pkgCacheGenerator::hash: Do not call tolower_ascii()Julian Andres Klode
Gbp-Dch: ignore
2016-01-08pkgCacheGenerator::StoreString: Get rid of std::stringJulian Andres Klode
Instead of storing a string -> map_stringitem_t mapping, create our own data type that can point to either a normal string or a string inside the cache. This avoids the creation of any string and improves performance slightly (about 4%).
2016-01-08Replace compare() == 0 checks with this == other checksJulian Andres Klode
This improves performance, as we now can ignore unequal strings based on their length already. Gbp-Dch: ignore
2016-01-08pkgCacheGenerator: Use StringView for toStringJulian Andres Klode
This removes some minor overhead. Gbp-Dch: ignore
2016-01-08pkgCacheGenerator::StoreString: Move the string into the mapJulian Andres Klode
Moving the string is likely faster than copying it. We could probably avoid strings alltogether in the future using some more crazy code, but I have not looked at that yet. Gbp-Dch: ignore
2016-01-07StringView: rfind: pos should be end of substr, not startJulian Andres Klode
Gbp-Dch: ignore
2016-01-07StringView: pos argument default should be nposJulian Andres Klode
Gbp-Dch: ignore
2016-01-07StringView::rfind(): Call rfind() instead of find() on substJulian Andres Klode
Thanks: Niels Thykier for reporting on IRC Gbp-Dch: ignore
2016-01-07Switch performance critical code to use APT::StringViewJulian Andres Klode
This improves performance of the cache generation on my ARM platform (4x Cortex A15) by about 10% to 20% from 2.35-2.50 to 2.1 seconds.
2016-01-07Introduce internal APT::StringView classJulian Andres Klode
The class APT::StringView implements a drop-in replacement for a subset of C++17 std::string_view() features. It will be dropped at a later point and may not be used in public interfaces.
2016-01-07rred: Run in parallelJulian Andres Klode
Remove the SingleInstance flag so we can use the new randomized queue feature to run parallel.
2016-01-07acquire: Allow parallelizing methods without hostsJulian Andres Klode
The maximum parallelization soft limit is the number of CPU cores * 2 on systems defining _SC_NPROCESSORS_ONLN. The hard limit in all cases is Acquire::QueueHost::Limit.
2016-01-07CopyFile: Use 64 * 1024 instead of 64000 as buffer sizeJulian Andres Klode
This is a multiple of the page size and thus results in less page faults, speeding up copying. Also, while we're at at, unify all uses of that size in a constant variable APT_BUFFER_SIZE.
2016-01-07apt-helper: cat-file: Add -C/--compress optionJulian Andres Klode
This allows passing compressing the output. The compressor must be a compressor name, extension, or an extension without the leading dot.
2016-01-07FileFd: (native) LZ4 supportJulian Andres Klode
Implement native support for LZ4 compression, using the official lz4 library.
2016-01-07travis: pull liblz4-dev from wilyJulian Andres Klode
2016-01-05Do not remove a not working SrvRecords server twiceMichael Vogt
The PopFromSrvRecs() already removed the entry from the active list, so the extra SrvRecords.erase() was incorrect. Git-Dch: ignore
2016-01-05Add documentation about the SrvRecords support in aptMichael Vogt
Git-Dch: ignore
2016-01-05Document new APT::Keep-Downloaded-Packages option in NEWSMichael Vogt
Git-Dch: ignore
2016-01-03test-apt-cache: Adjust for hashtable size changeJulian Andres Klode
Gbp-Dch: ignore
2016-01-03Increase APT::Cache-HashTableSize default to 50503Julian Andres Klode
This drop the hash table utilization from a high 98% to acceptable 74% on unstable, and the average bucket length from 4.6 to 1.8. This improves performance by about 5%, while increasing the size of the cache by 0.2 out of 38MB, that is 0.5%. 48481 is a nice number
2016-01-03apt-cache: stats: Show a table utilization as percentageJulian Andres Klode
Gbp-Dch: ignore
2016-01-03apt-cache: stats: Average is over used, not all, bucketsJulian Andres Klode
It does not make sense to consider empty buckets in the average, as they do not affect the lookup performance.
2016-01-03Allow building without libgtest-dev under <nocheck> build profileHelmut Grohne
I'd like to avoid pulling libgtest-dev into the bootstrap set. Fortunately, libgtest-dev is only used for testing apt and apt correctly implements DEB_BUILD_OPTIONS=nocheck now. So this bug is about getting rid of the Build-Depends. Simply removing it (by adding a build profile) is not sufficient however as configure fails hard, so an additional bit is necessary to cover for that. Closes: #809726
2016-01-03Change compressor costs to be 100 apartJulian Andres Klode
This will give us the freedom to insert more compressors at positions in between. Also change the cost of uncompressed to 0, as that really has no overhead, and the values do not really mean much.
2016-01-03simple_compressor: Provide some accessors for end and freeJulian Andres Klode
This makes code easier to read, and somewhat more correct. Gbp-Dch: ignore
2016-01-03simple_buffer: Allow buffer size to changeJulian Andres Klode
Gbp-Dch: ignore