| Age | Commit message (Collapse) | Author |
|---|
|
zstd is a compression algorithm developed by facebook. At level 19,
it is about 6% worse in size than xz -6, but decompression is multiple
times faster, saving about 40% install time, especially with eatmydata
on cloud instances.
|
|
A libapt user who hasn't initialized _system likely has a reason, so we
shouldn't greet back with a segfault usually deep down in the callstack
for no reason. If the user had intended to pick up information from the
system, _system wouldn't be uninitialized after all.
LP: #1613184
SRU: 1.4.y
|
|
This makes it easier to see which headers includes what.
The changes were done by running
git grep -l '#\s*include' \
| grep -E '.(cc|h)$' \
| xargs sed -i -E 's/(^\s*)#(\s*)include/\1#\2 include/'
To modify all include lines by adding a space, and then running
./git-clang-format.sh.
|
|
This is in so far pointless as the first match will deal with the
extension, so we don't actually ever use these second instances –
probably for the better as most need arguments to behave as epected &
more importantly: the point of the exercise disabling their use for
testing proposes.
Gbp-Dch: Ignore
|
|
We usually use absolute paths to specific the location of dpkg, apt-key
and the like, but there is nothing wrong with using just the command
name and instead let exec(3) make the lookup in PATH.
We had a wild mixture before, so opting for the more accepting option
out of the two seems about right especially as it makes no difference in
the default case as apt uses absolute paths.
|
|
libapt allows to configure compressors to be used by its system via
configuration implemented in 03bef78461c6f443187b60799402624326843396,
but that was never really documented and also only partly working, which
also explains why the tests weren't using it…
|
|
Downloading and storing are two different operations were different
compression types can be preferred. For downloading we provide the
choice via Acquire::CompressionTypes::Order as there is a choice to
be made between download size and speed – and limited by whats available
in the repository.
Storage on the other hand has all compressions currently supported by
apt available and to reduce runtime of tools accessing these files the
compression type should be a low-cost format in terms of decompression.
apt traditionally stores its indexes uncompressed on disk, but has
options to keep them compressed. Now that apt downloads additional files
we also deal with files which simply can't be stored uncompressed as
they are just too big (like Contents for apt-file). Traditionally they
are downloaded in a low-cost format (gz) as repositories do not provide
other formats, but there might be even lower-cost formats and for
download we could introduce higher-cost in the repositories.
Downloading an entire index potentially requires recompression to
another format, so an update takes potentially longer – but big files
are usually updated via pdiffs which has to de- and re-compress anyhow
and does it on the fly anyhow, so there is no extra time needed and in
general it seems to be benefitial to invest the time in update to save
time later on file access.
|
|
Implement native support for LZ4 compression, using the official
lz4 library.
|
|
This will give us the freedom to insert more compressors at
positions in between.
Also change the cost of uncompressed to 0, as that really has
no overhead, and the values do not really mean much.
|
|
Since commit 7a68effcb904b4424b54a30e448b6f2560cd1078, the xz
and lzma compressors read the level of compression they shall
use.
A default of -9 is too much for them, this will use 674 MB,
according to the xz manual page. Level -6 on the other hand
only needs 94 MB memory for compression.
This causes autopkgtest failures in the test-compressed-indexes
test, as not enough memory exists to proceed.
Change the other compression levels to 6 as well: The gzip
and bzip2 FileFd backends do not read them, and use their
code's default level which is 6, so do the same for external
methods.
|
|
|
|
strtok() is not thread-safe, whereas strtok_r() is.
Gbp-Dch: ignore
|
|
xz has pretty much won "the compressor war" and e.g. the Debian archive
doesn't even distribute bz2 anymore in favor of 'xz' and 'gz', so by
changing the default order we have a more realistic --print-uris
behavior as it will always show the first compressor.
In practice this effects repositories without a Release file (very bad,
we don't want to support them anymore anyhow) as xz will be tried before
bz2 now [which is probably not available, but so might be bz2…] AND
repositories which provide both, bz2 and xz (which isn't too common) in
sofar as apt will now download xz instead of bz2.
Users with special needs can stick with bz2 as first compressor tried
with Acquire::CompressionTypes::Order:: "bz2"; (see man apt.conf) – but
users with special needs usually prefer "gz" anyhow, so the realworld
change is expected to be very low.
|
|
Nobody knows what makes the 'unable to optimize loop' warning to appear
in the sourceslist minus-options parsing, especially if we use a foreach
loop, but we can replace it with some nice c++11 algorithm+lambda usage,
which also helps in making even clearer what happens here.
And as this would be a lonely change, lets do it for a few more loops as
well where I might or might not have seen the warning at some point in
time, too.
Git-Dch: Ignore
|
|
|
|
This allows us to run the clang static analyzer and to run the
testsuite with the clang MemorySanitizer.
|
|
We are the only possible users of private methods, so we are also the
only users who can potentially export them via using them in inline
methods. The point is: We don't need these symbols exported if we don't
do this, so marking them as hidden removes some methods from the API
without breaking anything as nobody could have used them.
Git-Dch: Ignore
|
|
The cache heavily depends on the architecture(s) it is build for,
especially if you move from single- to multiarch. Adding a new
architecture to dpkg therefore has to be detected and must invalidate
the cache so that we don't operate on incorrect data.
The incorrect data will prevent us from doing otherwise sensible
actions (it doesn't allow bad things to happen) and the recovery is
simple and automatic in most cases, so this hides pretty well and is
also not as serious as it might sound at first.
Closes: 745036
|
|
FileFd code knows how to deal with such a compressor, so it isn't a
problem, but it is absolutely not needed as we already have an
(matching) identity compressor with '.' earlier in the list.
Git-Dch: Ignore
|
|
We have xz/lzma support for a while, but only via an external binary
provided by xz-utils. Now that the Debian archive provides xz by default
and dpkg pre-depends on the library provided by liblzma-dev we can switch
now to use this library as well to avoid requiring an external binary.
For now the binary is in a prio:required package, but this might change
in the future.
API wise it is quiet similar to bz2 code expect that it doesn't provide
file I/O methods, so we piece this together on our own.
|
|
Beside being a bit cleaner it hopefully also resolves oddball problems
I have with high levels of parallel jobs.
Git-Dch: Ignore
Reported-By: iwyu (include-what-you-use)
|
|
Reported-By: gcc -Wignored-qualifiers
Git-Dch: Ignore
|
|
Inspired by the rest of the patch in 661537, but abstract the
parsing of various ways of setting the build profiles more so it can
potentially be reused and all apt parts have the same behaviour.
Especially config options, cmdline options and environment will not be
combined as proposed as this isn't APTs usual behaviour and dpkg doesn't
do it either, so one overrides the other as it normally does.
|
|
Automatically handle the override of list options via its parent value
which can even be a comma-separated list of values. It also adds an easy
way of providing a default for the list.
|
|
|
|
No visible functional changes, just code moved around and additional
checks to eliminate impossible branches
Reported-By: scan-build
Git-Dch: Ignore
|
|
Clear() only clears a config option, not removing it and an empty
setting still exists. Hence we set the option instead to the xz path
so that the later existance check can find a binary for the test
|
|
|
|
|
|
|
|
- copy only configured translation files from a CD-ROM and not all
available translation files preventing new installs with d-i from
being initialized with all translations (Closes: #678227)
- handle Components in the reduction for the source.list as multi-arch CDs
otherwise create duplicated source entries (e.g. "wheezy main main")
|
|
- if APT::Languages=none save "none" in allCodes so that the detected
configuration is cached as intended (Closes: #674690, LP: #1004947)
|
|
- longcode Translation files are saved with encoded underscore,
so make sure to pick these files up as well for Acquire::Languages
|
|
- use NULL instead of "" for no (un)compress parameters
|
|
preference given by the Order setting is really available before adding
it as possible CompressionType.
|
|
- if the compressor is not installed, but we link against it's
library accept it as a CompressionType (Closes: #669328)
|
|
ignore the presents (or absence) of lzma if we decided to use xz
|
|
- remove the libz-dev alternative from zlib1g-dev build-dependency
- do the same for bz2 builtin if available
* apt-pkg/contrib/fileutl.cc:
- use libz2 library for (de)compression instead of the bzip2 binary as
the first is a dependency of dpkg and the later just priority:optional
so we gain 'easier' access to bz2-compressed Translation files this way
|
|
|
|
- if present, prefer xz binary over lzma
|
|
|
|
|
|
- chroot if needed before calling dpkg --print-foreign-architectures
|
|
- parse dpkg --print-foreign-architectures correctly in
case archs are separated by newline instead of space, too.
(Closes: #655590)
* Slovak (Ivan Masar). Closes: #652985
* Russian (Yuri Kozlov). Closes: #654844
* Hungarian (Gabor Kelemen). Closes: #655238
|
|
- parse dpkg --print-foreign-architectures correctly in
case archs are separated by newline instead of space, too.
|
|
* apt-pkg/deb/debmetaindex.cc:
- none is a separator, not a language: no need for Index (Closes: #624218)
* apt-pkg/aptconfiguration.cc:
- do not builtin languages only if none is forced (Closes: #643787)
|
|
- none is a separator, not a language: no need for Index (Closes: #624218)
* apt-pkg/aptconfiguration.cc:
- do not builtin languages only if none is forced (Closes: #643787)
|
|
|
|
|
|
|
