| Age | Commit message (Collapse) | Author |
|---|
|
Strings like Section names or architectures are needed vary often.
Instead of writing them each time we need them, we deploy sharing for
these special strings. Until now, this was done with a linked list of
strings in which we would search, which was stored in the cache.
It turns out we can do this just as well in memory as well with a bunch
of std::map's.
In memory means here that it isn't available anymore if we have a partly
invalid cache, but that isn't much of a problem in practice as the
status file is compared to the other files we parse very small and includes
mostly duplicates, so the space we would gain by storing is more or less
equal to the size of the stored linked list…
|
|
Commit aa0fe657e46b87cc692895a36df12e8b74bb27bb sorts the package names
in the hashtable. We make use of this already in these functions, but as
a minor sideeffect it also means that we don't have 'noise' anymore
between packages belonging to the same group. We therefore don't need to
check for a matching name in Grp.FindPkg anymore.
Git-Dch: Ignore
|
|
Package names have to be lowercase (debian-policy §5.6.1) and in as
lowlevel as these method are it would be quiet strange to treat an
invalid package "suddently" as a valid one which other tools might or
might not accept. If case-insensitivity is really needed the frontend
should ensure this rather than these methods waste cpu cycles by
default.
Git-Dch: Ignore
|
|
They both store the same information, so this field just takes up space
in the Package struct for no good reason. We mark it "just" as deprecated
instead of instantly removing it though as it isn't misleading like
Section was and is potentially used in the wild more often.
|
|
A version belongs to a section and has hence a section member of its
own. A package on the other hand can have multiple versions from
different sections. This was "solved" by using the section which was
parsed first as order of sources.list defines, but that is obviously a
horribly unpredictable thing.
We therefore directly remove this struct member to free some space and
mark the access method as deprecated, which is told to return the
section of the 'newest' known version, which is at least predictable,
but possible not what it returned before – but nobody knows.
Users are way better of with the Section() as returned by the version
they are dealing with. It is likely the same for all versions of a
package, but in the few cases it isn't, it is important (like packages
moving from main/* to contrib/* or into oldlibs …).
|
|
We had a wild mixture of (unsigned) int, long and long long here without
much sense, so this commit adds a few typedefs to get some sense in the
typesystem and ensures that a ID isn't sometimes computed as int, stored
as long and compared with a long long… as this could potentially bite us
later on as the size of the archive only increases over time.
|
|
It also makes the size configureable, so it can be adapted in the future
without the need for an abi break - and even by users…
The increase was long overdue as it gives a >10% decrease in runtime of
e.g. 'apt-get check -s'. Some (useless) benchmark with 69933 groups and
187796 packages without a pre-built cache:
time apt-get check -so APT::Cache-HashTableSize=1 → 20m
time apt-get check -so APT::Cache-HashTableSize=1000 → 6,41s
time apt-get check -so APT::Cache-HashTableSize=2000 → 5,64s (old)
time apt-get check -so APT::Cache-HashTableSize=3000 → 5,30s
time apt-get check -so APT::Cache-HashTableSize=5000 → 5,08s
time apt-get check -so APT::Cache-HashTableSize=6000 → 5,05s
time apt-get check -so APT::Cache-HashTableSize=7000 → 5,02s
time apt-get check -so APT::Cache-HashTableSize=8000 → 5,00s
time apt-get check -so APT::Cache-HashTableSize=9000 → 4,98s
time apt-get check -so APT::Cache-HashTableSize=10000 → 4,96s (new)
time apt-get check -so APT::Cache-HashTableSize=15000 → 4,90s
time apt-get check -so APT::Cache-HashTableSize=20000 → 4,86s
time apt-get check -so APT::Cache-HashTableSize=30000 → 4,77s
time apt-get check -so APT::Cache-HashTableSize=40000 → 4,74s
time apt-get check -so APT::Cache-HashTableSize=50000 → 4,73s
time apt-get check -so APT::Cache-HashTableSize=60000 → 4,71s
The gap increases further for operations which have more package
lookups. Factor 5 was chosen as higher values do not provide any
really significant timing advantage anymore compared to the memory
increase in my testing and there is always the possibility to increase
it now if that changes. (also most users will not have 3 releases and
4 architectures in the cache, so theirs will be much smaller and faster).
|
|
Conflicts:
apt-pkg/acquire-item.cc
apt-pkg/acquire-item.h
apt-pkg/deb/debmetaindex.h
apt-pkg/pkgcache.cc
test/integration/test-apt-ftparchive-src-cachedb
|
|
This is a internal struct not a external interface so the actual
breakage should be small.
|
|
|
|
Git-Dch: Ignore
Reported-By: gcc
|
|
The cache heavily depends on the architecture(s) it is build for,
especially if you move from single- to multiarch. Adding a new
architecture to dpkg therefore has to be detected and must invalidate
the cache so that we don't operate on incorrect data.
The incorrect data will prevent us from doing otherwise sensible
actions (it doesn't allow bad things to happen) and the recovery is
simple and automatic in most cases, so this hides pretty well and is
also not as serious as it might sound at first.
Closes: 745036
|
|
Git-Dch: Ignore
Reported-By: gcc
|
|
Conflicts:
apt-pkg/cachefilter.h
apt-pkg/contrib/fileutl.cc
apt-pkg/contrib/netrc.h
apt-pkg/deb/debsrcrecords.cc
apt-pkg/init.h
apt-pkg/pkgcache.cc
debian/apt.install.in
debian/changelog
|
|
In #737085 we see that apt can be confused if informations about
versions only differ slightly. This commit adds a way of at least adding
a few more data points with the next abi break to help a bit with it.
Git-Dch: Ignore
|
|
Beside being a bit cleaner it hopefully also resolves oddball problems
I have with high levels of parallel jobs.
Git-Dch: Ignore
Reported-By: iwyu (include-what-you-use)
|
|
Reported-By: gcc -Wunused-parameter
Git-Dch: Ignore
|
|
Git-Dch: Ignore
Reported-By: gcc -Wpedantic
|
|
Conflicts:
apt-private/private-list.cc
configure.ac
debian/apt.install.in
debian/changelog
|
|
|
|
Conflicts:
apt-pkg/contrib/strutl.cc
apt-pkg/deb/dpkgpm.cc
configure.ac
debian/changelog
doc/po/apt-doc.pot
po/apt-all.pot
po/ar.po
po/ast.po
po/bg.po
po/bs.po
po/ca.po
po/cs.po
po/cy.po
po/da.po
po/de.po
po/dz.po
po/el.po
po/es.po
po/eu.po
po/fi.po
po/fr.po
po/gl.po
po/hu.po
po/it.po
po/ja.po
po/km.po
po/ko.po
po/ku.po
po/lt.po
po/mr.po
po/nb.po
po/ne.po
po/nl.po
po/nn.po
po/pl.po
po/pt.po
po/pt_BR.po
po/ro.po
po/ru.po
po/sk.po
po/sl.po
po/sv.po
po/th.po
po/tl.po
po/uk.po
po/vi.po
po/zh_CN.po
po/zh_TW.po
test/integration/framework
test/integration/test-bug-602412-dequote-redirect
test/integration/test-ubuntu-bug-346386-apt-get-update-paywall
test/interactive-helper/aptwebserver.cc
test/interactive-helper/makefile
|
|
Adds on top of Version 2 to all displayed version numbers the
architecture as well as the MultiArch flag for consumption by the hooks.
Most of the time the architecture will be the same for both versions
displayed, but packages might change from "all" to "any" (or back)
between versions so we can't display the architecture for packages.
Pseudo-Format for Version 3:
<name> <version> <arch> <m-a-flag> <compare> <version> <arch> <m-a-flag>
Examples:
stuff - - none < 1 amd64 none **CONFIGURE**
libsame 1 i386 same < 2 i386 same **CONFIGURE**
stuff 2 i386 none > 1 i386 none **CONFIGURE**
libsame 2 i386 same > - - none **REMOVE**
toolkit 1 all foreign > - - none **REMOVE**
Closes: #712116
|
|
|
|
- provide DepIterator::IsSatisfied as a nicer shorthand for DepCheck
|
|
* apt-pkg/pkgcache.cc:
- assume sorted hashtable entries for groups/packages
|
|
|
|
- adjust pkgCache::State::VerPriority enum, to match reality
|
|
- ignore negative dependencies applying in the same group for
M-A:same packages on the real package name as self-conflicts
|
|
by introducing a pseudo-architecture 'none' so that the small group of
users with these packages can get right of them without introducing too
much hassle for other users (Closes: #686346)
|
|
- do a string comparision for architecture checking in IsMultiArchImplicit
as 'unique' strings in the pkgcache aren't unique (Closes: #677454)
|
|
- add support for arch-specific qualifiers in dependencies
|
|
- add an IsMultiArchImplicit() method for Dep- and PrvIterator
|
|
|
|
- ignore implicit conflicts on providers in AllTarget, too
|
|
- always prefer "en" over "" for "en"-language regardless of cache-order
|
|
|
|
|
|
|
|
|
|
- ignore "self"-conflicts for all architectures of a package
instead of just for the architecture of the package locked at
in the ordering of installations too (Closes: #802901)
|
|
|
|
done on the mirco-optimazation level, so lets fix them:
(performance) Possible inefficient checking for emptiness.
(performance) Prefer prefix ++/-- operators for non-primitive types.
|
|
- Check that cache is at least CacheFileSize bytes large (LP: #16467)
|
|
|
|
pseudo-package related and/or private
|
|
for the three different dependencies
|
|
|
|
- really ignore :arch in FindPkg() in non-multiarch environment
|
|
- use the native Architecture stored in the cache header instead of
loading it from configuration as suggested by Julian Andres Klode
|
|
that it is a needed "hack" to support multi-arch in the least breaking way
|
