| Age | Commit message (Collapse) | Author |
|---|
|
We used to read the Release file for each Packages file and store the
data in the PackageFile struct even through potentially many Packages
(and Translation-*) files could use the same data. The point of the
exercise isn't the duplicated data through. Having the Release files as
first-class citizens in the Cache allows us to properly track their
state as well as allows us to use the information also for files which
aren't in the cache, but where we know to which Release file they
belong (Sources are an example for this).
This modifies the pkgCache structs, especially the PackagesFile struct
which depending on how libapt users access the data in these structs can
mean huge breakage or no visible change. As a single data point:
aptitude seems to be fine with this. Even if there is breakage it is
trivial to fix in a backportable way while avoiding breakage for
everyone would be a huge pain for us.
Note that not all PackageFile structs have a corresponding ReleaseFile.
In particular the dpkg/status file as well as *.deb files have not. As
these have only a Archive property need, the Component property takes
over this duty and the ReleaseFile remains zero. This is also the reason
why it isn't needed nor particularily recommended to change from
PackagesFile to ReleaseFile blindly. Sticking with the earlier is
usually the better option.
|
|
Strings are easy to typo and we can keep the extensibility we require
here with a simple enum we can append to without endangering ABI.
Git-Dch: Ignore
|
|
Removes a bunch of duplicated code in the deb-specific parts. Especially
the Description part is now handled centrally by IndexTarget instead of
being duplicated to the derivations of IndexFile.
Git-Dch: Ignore
|
|
It is a rather strange sight that index items use SiteOnly which strips
the Path, while e.g. deb files are downloaded with NoUserPassword which
does not. Important to note here is that for the file transport Path is
pretty important as there is no Host which would be displayed by Site,
which always resulted in "interesting" unspecific errors for "file:".
Adding a 'middle' ground between the two which does show the Path but
potentially modifies it (it strips a pending / at the end if existing)
solves this "file:" issue, syncs the output and in the end helps to
identify which file is meant exactly in progress output and co as a
single site can have multiple repositories in different paths.
|
|
Having every item having its own code to verify the file(s) it handles
is an errorprune process and easy to break, especially if items move
through various stages (download, uncompress, patching, …). With a giant
rework we centralize (most of) the verification to have a better
enforcement rate and (hopefully) less chance for bugs, but it breaks the
ABI bigtime in exchange – and as we break it anyway, it is broken even
harder.
It shouldn't effect most frontends as they don't deal with the acquire
system at all or implement their own items, but some do and will need to
be patched (might be an opportunity to use apt on-board material).
The theory is simple: Items implement methods to decide if hashes need to
be checked (in this stage) and to return the expected hashes for this
item (in this stage). The verification itself is done in worker message
passing which has the benefit that a hashsum error is now a proper error
for the acquire system rather than a Done() which is later revised to a
Failed().
|
|
In 66c3875df391b1120b43831efcbe88a78569fbfe we workaround/fixed a
problem where the code makes the assumption that the compiler uses
copy-on-write implementations for std::string. Turns out that for c++11
compatibility gcc >= 5 will stop doing this by default.
|
|
Implementing FileName() works for most cases for us, but other
frontends might need more and even for us its not very stable as
the normal Jump() implementation is pretty bad on a deb file and
produce errors on its own at times.
So, replacing this makeshift with a complete implementation by
mostly just shuffling code around.
|
|
We have a bunch of classes which are of no use for the outside world,
but were still exported and so needed to preserve ABI/API. Marking them
as hidden to not export them any longer is a big API break in theory,
but in practice nobody is using them – as if they would its a bug.
|
|
|
|
Handle Translation-* files exactly like Packages files (with the
expection that it is ok if a download of them fails). Remove all
"guessing" on apts side. This will elimimnate a bunch of errors
releated to captive portals and similar. Its also more correct
and removes another potential attack vector.
|
|
Reimplementing an inline method is opening a can of worms we don't want
to open if we ever want to us a d-pointer in those classes, so we do the
only thing which can save us from hell: move the destructors into the cc
sources and we are good.
Technically not an ABI break as the methods inline or not do the same
(nothing), so a program compiled against the old version still works
with the new version (beside that this version is still in experimental,
so nothing really has been build against this library anyway).
Git-Dch: Ignore
|
|
We are the only possible users of private methods, so we are also the
only users who can potentially export them via using them in inline
methods. The point is: We don't need these symbols exported if we don't
do this, so marking them as hidden removes some methods from the API
without breaking anything as nobody could have used them.
Git-Dch: Ignore
|
|
Conflicts:
apt-pkg/acquire-item.cc
|
|
Strings like Section names or architectures are needed vary often.
Instead of writing them each time we need them, we deploy sharing for
these special strings. Until now, this was done with a linked list of
strings in which we would search, which was stored in the cache.
It turns out we can do this just as well in memory as well with a bunch
of std::map's.
In memory means here that it isn't available anymore if we have a partly
invalid cache, but that isn't much of a problem in practice as the
status file is compared to the other files we parse very small and includes
mostly duplicates, so the space we would gain by storing is more or less
equal to the size of the stored linked list…
|
|
|
|
CLoses: #752327
|
|
We had a wild mixture of (unsigned) int, long and long long here without
much sense, so this commit adds a few typedefs to get some sense in the
typesystem and ensures that a ID isn't sometimes computed as int, stored
as long and compared with a long long… as this could potentially bite us
later on as the size of the archive only increases over time.
|
|
Conflicts:
test/integration/test-bug-747261-arch-specific-conflicts
|
|
Version/Architecture information in a Translation-$lang file is
not allowed, so don't try to parse it. This is a fix for a bugreport
where a Translation-en file contained the content of the regular
Packages file (probably due to local FS corruption). This lead to
strange error messages on file download.
Thanks to Thomas Reusch for the report.
|
|
|
|
debian/experimental
Conflicts:
apt-pkg/deb/debindexfile.cc
apt-pkg/deb/debindexfile.h
apt-pkg/deb/debsrcrecords.cc
|
|
|
|
|
|
|
|
|
|
|
|
Git-Dch: Ignore
Reported-By: gcc -Wsuggest-attribute={pure,const,noreturn}
|
|
Beside being a bit cleaner it hopefully also resolves oddball problems
I have with high levels of parallel jobs.
Git-Dch: Ignore
Reported-By: iwyu (include-what-you-use)
|
|
apt-pkg/deb/deblistparser.cc:
- use OpenMaybeClearSignedFile to be free from detecting and
skipping clearsigning metadata in dsc and Release files
We can't write a "clean" file to disk as not all acquire methods copy
Release files before checking them (e.g. cdrom), so this reverts recombining,
but uses the method we use for dsc files also in the two places we
deal with Release files
|
|
change at the time of writing the strings, so first store it temporary
and then save the index in the (possibily new) pointer location
|
|
|
|
on the FileFd instead
|
|
(and be it if it tries to announce that…)
|
|
- implement a ModificationTime method for FileFd
|
|
to search for compressed silbings of the given filename and use this guessing
instead of hardcoding Gzip compression
|
|
|
|
|
|
done on the mirco-optimazation level, so lets fix them:
(performance) Possible inefficient checking for emptiness.
(performance) Prefer prefix ++/-- operators for non-primitive types.
|
|
that is not needed
|
|
this will break adding translations to the cache if the current environment does not include the language (e.g. LANG=C but german translations). testing for existance of the file is the better approach
|
|
- ignore missing deb-src files in /var/lib/apt/lists, thanks
to Thorsten Spindler (LP: #85590)
|
|
* apt-pkg/deb/deblistparser.cc:
- rewrite LoadReleaseInfo to cope with clearsigned Releasefiles
|
|
|
|
- Use FileFd::Size() instead of stat()ing the sources/binary/translations
indexes directly, so that we have transparent handling of gzipped
indexes.
|
|
- do not query each architecture for flat file archives
|
|
* debian/rules:
- Make DEB_BUILD_OPTIONS=noopt actually work by passing the right
CXXFLAGS.
* apt-pkg/contrib/fileutl.{h,cc}:
- Add support for reading of gzipped files with the new "ReadOnlyGzip"
OpenMode. (Closes: #188407)
- Link against zlib (in apt-pkg/makefile) and add zlib build dependency.
- [ABI BREAK] This adds a new private member to FileFd, but its
initialization is in the public header file.
* configure.in:
- Check for zlib library and headers.
* apt-pkg/acquire-item.cc, apt-pkg/deb/debindexfile.cc,
apt-pkg/deb/debrecords.cc, apt-pkg/deb/debsrcrecords.h,
cmdline/apt-cache.cc:
- Open Packages, Sources, and Translations indexes in "ReadOnlyGzip" mode.
* apt-pkg/deb/debindexfile.cc:
- If we do not find uncompressed package/source/translation indexes, look
for gzip compressed ones.
* apt-pkg/acquire-item.cc:
- If the Acquire::GzipIndexes option is true and we download a gzipped
index file, keep it as it is (and rename to .gz) instead of
uncompressing it.
* doc/apt.conf.5.xml:
- Document the new Acquire::GzipIndexes option.
* doc/po/apt-doc.pot, doc/po/de.po:
- German translation of new Acquire::GzipIndexes option.
* Add test/test-indexes.sh:
- Test behaviour of index retrieval and usage, in particular with
uncompressed and gzip compressed indexes.
* methods/gzip.cc: With FileFd now being able to read gzipped files, there
is no need for the gzip method any more to spawn an external gzip process.
Rewrite it to use FileFd directly, which makes the code a lot simpler, and
also using less memory and overhead.
|
|
- make the used MMap moveable (and therefore dynamic resizeable) by
applying (some) mad pointer magic (Closes: #195018)
|
|
expects the compressed contents to stay (such as the copy backend, or when using file:// repositories. Instead, introduce a new ReadOnlyGzip mode and use that where needed
|
|
indexes, to work with apt-get source as well
|
|
- If we do not find uncompressed package/source/translation indexes, look
for gzip compressed ones.
|
