summaryrefslogtreecommitdiff
path: root/apt-pkg/deb/debindexfile.cc
AgeCommit message (Collapse)Author
2015-06-12store Release files data in the CacheDavid Kalnischkies
We used to read the Release file for each Packages file and store the data in the PackageFile struct even through potentially many Packages (and Translation-*) files could use the same data. The point of the exercise isn't the duplicated data through. Having the Release files as first-class citizens in the Cache allows us to properly track their state as well as allows us to use the information also for files which aren't in the cache, but where we know to which Release file they belong (Sources are an example for this). This modifies the pkgCache structs, especially the PackagesFile struct which depending on how libapt users access the data in these structs can mean huge breakage or no visible change. As a single data point: aptitude seems to be fine with this. Even if there is breakage it is trivial to fix in a backportable way while avoiding breakage for everyone would be a huge pain for us. Note that not all PackageFile structs have a corresponding ReleaseFile. In particular the dpkg/status file as well as *.deb files have not. As these have only a Archive property need, the Component property takes over this duty and the ReleaseFile remains zero. This is also the reason why it isn't needed nor particularily recommended to change from PackagesFile to ReleaseFile blindly. Sticking with the earlier is usually the better option.
2015-06-11use an enum instead of strings as IndexTarget::Option interfaceDavid Kalnischkies
Strings are easy to typo and we can keep the extensibility we require here with a simple enum we can append to without endangering ABI. Git-Dch: Ignore
2015-06-11use IndexTarget to get to IndexFileDavid Kalnischkies
Removes a bunch of duplicated code in the deb-specific parts. Especially the Description part is now handled centrally by IndexTarget instead of being duplicated to the derivations of IndexFile. Git-Dch: Ignore
2015-06-11show URI.Path in all acquire item descriptionsDavid Kalnischkies
It is a rather strange sight that index items use SiteOnly which strips the Path, while e.g. deb files are downloaded with NoUserPassword which does not. Important to note here is that for the file transport Path is pretty important as there is no Host which would be displayed by Site, which always resulted in "interesting" unspecific errors for "file:". Adding a 'middle' ground between the two which does show the Path but potentially modifies it (it strips a pending / at the end if existing) solves this "file:" issue, syncs the output and in the end helps to identify which file is meant exactly in progress output and co as a single site can have multiple repositories in different paths.
2015-06-09rework hashsum verification in the acquire systemDavid Kalnischkies
Having every item having its own code to verify the file(s) it handles is an errorprune process and easy to break, especially if items move through various stages (download, uncompress, patching, …). With a giant rework we centralize (most of) the verification to have a better enforcement rate and (hopefully) less chance for bugs, but it breaks the ABI bigtime in exchange – and as we break it anyway, it is broken even harder. It shouldn't effect most frontends as they don't deal with the acquire system at all or implement their own items, but some do and will need to be patched (might be an opportunity to use apt on-board material). The theory is simple: Items implement methods to decide if hashes need to be checked (in this stage) and to return the expected hashes for this item (in this stage). The verification itself is done in worker message passing which has the benefit that a hashsum error is now a proper error for the acquire system rather than a Done() which is later revised to a Failed().
2015-05-11stop depending on copy-on-write for std::stringDavid Kalnischkies
In 66c3875df391b1120b43831efcbe88a78569fbfe we workaround/fixed a problem where the code makes the assumption that the compiler uses copy-on-write implementations for std::string. Turns out that for c++11 compatibility gcc >= 5 will stop doing this by default.
2015-03-16properly implement pkgRecord::Parser for *.deb filesDavid Kalnischkies
Implementing FileName() works for most cases for us, but other frontends might need more and even for us its not very stable as the normal Jump() implementation is pretty bad on a deb file and produce errors on its own at times. So, replacing this makeshift with a complete implementation by mostly just shuffling code around.
2014-11-08mark internal interfaces as hiddenDavid Kalnischkies
We have a bunch of classes which are of no use for the outside world, but were still exported and so needed to preserve ABI/API. Marking them as hidden to not export them any longer is a big API break in theory, but in practice nobody is using them – as if they would its a bug.
2014-11-05apt-pkg/deb/debindexfile.{cc,h}: kill GetIndexes()Michael Vogt
2014-10-29Only support Translation-* that are listed in the {In,}Release fileMichael Vogt
Handle Translation-* files exactly like Packages files (with the expection that it is ok if a download of them fails). Remove all "guessing" on apts side. This will elimimnate a bunch of errors releated to captive portals and similar. Its also more correct and removes another potential attack vector.
2014-10-13do not inline virtual destructors with d-pointersDavid Kalnischkies
Reimplementing an inline method is opening a can of worms we don't want to open if we ever want to us a d-pointer in those classes, so we do the only thing which can save us from hell: move the destructors into the cc sources and we are good. Technically not an ABI break as the methods inline or not do the same (nothing), so a program compiled against the old version still works with the new version (beside that this version is still in experimental, so nothing really has been build against this library anyway). Git-Dch: Ignore
2014-09-30mark private methods as hiddenDavid Kalnischkies
We are the only possible users of private methods, so we are also the only users who can potentially export them via using them in inline methods. The point is: We don't need these symbols exported if we don't do this, so marking them as hidden removes some methods from the API without breaking anything as nobody could have used them. Git-Dch: Ignore
2014-09-29Merge branch 'debian/sid' into debian/experimentalMichael Vogt
Conflicts: apt-pkg/acquire-item.cc
2014-09-27drop stored StringItems in favor of in-memory mappingsDavid Kalnischkies
Strings like Section names or architectures are needed vary often. Instead of writing them each time we need them, we deploy sharing for these special strings. Until now, this was done with a linked list of strings in which we would search, which was stored in the cache. It turns out we can do this just as well in memory as well with a bunch of std::map's. In memory means here that it isn't available anymore if we have a partly invalid cache, but that isn't much of a problem in practice as the status file is compared to the other files we parse very small and includes mostly duplicates, so the space we would gain by storing is more or less equal to the size of the stored linked list…
2014-09-21generalize Acquire::GzipIndexMichael Vogt
2014-07-10Allow passing a full path to apt-get install /foo/bar.debMichael Vogt
CLoses: #752327
2014-06-18cleanup datatypes mix used in binary cacheDavid Kalnischkies
We had a wild mixture of (unsigned) int, long and long long here without much sense, so this commit adds a few typedefs to get some sense in the typesystem and ensures that a ID isn't sometimes computed as int, stored as long and compared with a long long… as this could potentially bite us later on as the size of the archive only increases over time.
2014-05-29Merge remote-tracking branch 'upstream/debian/sid' into debian/experimentalMichael Vogt
Conflicts: test/integration/test-bug-747261-arch-specific-conflicts
2014-05-15Never parse Version/Architecture tags in a Translation-$lang fileMichael Vogt
Version/Architecture information in a Translation-$lang file is not allowed, so don't try to parse it. This is a fix for a bugreport where a Translation-en file contained the content of the regular Packages file (probably due to local FS corruption). This lead to strange error messages on file download. Thanks to Thomas Reusch for the report.
2014-05-08apt-pkg/deb/debindexfile.cc: do not hardcode dpkgMichael Vogt
2014-05-08Merge remote-tracking branch 'mvo/feature/build-dep-dsc2' into ↵Michael Vogt
debian/experimental Conflicts: apt-pkg/deb/debindexfile.cc apt-pkg/deb/debindexfile.h apt-pkg/deb/debsrcrecords.cc
2014-04-28use flAbsPath()Michael Vogt
2014-04-28use new Popen()Michael Vogt
2014-04-28WIP local deb installMichael Vogt
2014-04-25add support for apt-get build-dep unpacked-source-dirMichael Vogt
2014-04-24create debIFTypeDscFile typeMichael Vogt
2014-03-13follow method attribute suggestions by gccDavid Kalnischkies
Git-Dch: Ignore Reported-By: gcc -Wsuggest-attribute={pure,const,noreturn}
2014-03-13cleanup headers and especially #includes everywhereDavid Kalnischkies
Beside being a bit cleaner it hopefully also resolves oddball problems I have with high levels of parallel jobs. Git-Dch: Ignore Reported-By: iwyu (include-what-you-use)
2013-03-18* apt-pkg/deb/debindexfile.cc,David Kalnischkies
apt-pkg/deb/deblistparser.cc: - use OpenMaybeClearSignedFile to be free from detecting and skipping clearsigning metadata in dsc and Release files We can't write a "clean" file to disk as not all acquire methods copy Release files before checking them (e.g. cdrom), so this reverts recombining, but uses the method we use for dsc files also in the two places we deal with Release files
2012-06-16do not dereference the storage for the unique strings as the pointer canDavid Kalnischkies
change at the time of writing the strings, so first store it temporary and then save the index in the (possibily new) pointer location
2012-05-10fix calling style of FileFd (no functional change)David Kalnischkies
2011-12-17try to avoid direct usage of .Fd() if possible and do read()s and coDavid Kalnischkies
on the FileFd instead
2011-12-15atleast libapt should announce to itself that it is clean…David Kalnischkies
(and be it if it tries to announce that…)
2011-12-10* apt-pkg/contrib/fileutl.{h,cc}:David Kalnischkies
- implement a ModificationTime method for FileFd
2011-12-10enable FileFd to guess the compressor based on the filename if requested orDavid Kalnischkies
to search for compressed silbings of the given filename and use this guessing instead of hardcoding Gzip compression
2011-09-13merge with debian/experimentalDavid Kalnischkies
2011-09-13reorder includes: add <config.h> if needed and include it at firstDavid Kalnischkies
2011-08-11cppcheck complains about some possible speed improvements which could beDavid Kalnischkies
done on the mirco-optimazation level, so lets fix them: (performance) Possible inefficient checking for emptiness. (performance) Prefer prefix ++/-- operators for non-primitive types.
2011-06-09apt-pkg/deb/debindexfile.cc: kill off another TranslationsAvailable() usage ↵Michael Vogt
that is not needed
2011-06-09apt-pkg/deb/debindexfile.cc: remove tests for TranslationsAvailable() as ↵Michael Vogt
this will break adding translations to the cache if the current environment does not include the language (e.g. LANG=C but german translations). testing for existance of the file is the better approach
2011-02-25* apt-pkg/deb/debindexfile.cc:Michael Vogt
- ignore missing deb-src files in /var/lib/apt/lists, thanks to Thorsten Spindler (LP: #85590)
2011-01-20 - try downloading clearsigned InRelease before trying Release.gpgDavid Kalnischkies
* apt-pkg/deb/deblistparser.cc: - rewrite LoadReleaseInfo to cope with clearsigned Releasefiles
2010-10-15add test case for having no indexes, and fix regression in that caseMartin Pitt
2010-10-13* apt-pkg/deb/debindexfile.cc:Martin Pitt
- Use FileFd::Size() instead of stat()ing the sources/binary/translations indexes directly, so that we have transparent handling of gzipped indexes.
2010-07-11* apt-pkg/deb/debmetaindex.cc:David Kalnischkies
- do not query each architecture for flat file archives
2010-07-11[ Martin Pitt ]David Kalnischkies
* debian/rules: - Make DEB_BUILD_OPTIONS=noopt actually work by passing the right CXXFLAGS. * apt-pkg/contrib/fileutl.{h,cc}: - Add support for reading of gzipped files with the new "ReadOnlyGzip" OpenMode. (Closes: #188407) - Link against zlib (in apt-pkg/makefile) and add zlib build dependency. - [ABI BREAK] This adds a new private member to FileFd, but its initialization is in the public header file. * configure.in: - Check for zlib library and headers. * apt-pkg/acquire-item.cc, apt-pkg/deb/debindexfile.cc, apt-pkg/deb/debrecords.cc, apt-pkg/deb/debsrcrecords.h, cmdline/apt-cache.cc: - Open Packages, Sources, and Translations indexes in "ReadOnlyGzip" mode. * apt-pkg/deb/debindexfile.cc: - If we do not find uncompressed package/source/translation indexes, look for gzip compressed ones. * apt-pkg/acquire-item.cc: - If the Acquire::GzipIndexes option is true and we download a gzipped index file, keep it as it is (and rename to .gz) instead of uncompressing it. * doc/apt.conf.5.xml: - Document the new Acquire::GzipIndexes option. * doc/po/apt-doc.pot, doc/po/de.po: - German translation of new Acquire::GzipIndexes option. * Add test/test-indexes.sh: - Test behaviour of index retrieval and usage, in particular with uncompressed and gzip compressed indexes. * methods/gzip.cc: With FileFd now being able to read gzipped files, there is no need for the gzip method any more to spawn an external gzip process. Rewrite it to use FileFd directly, which makes the code a lot simpler, and also using less memory and overhead.
2010-07-08* apt-pkg/pkgcachegen.{cc,h}:David Kalnischkies
- make the used MMap moveable (and therefore dynamic resizeable) by applying (some) mad pointer magic (Closes: #195018)
2010-06-24Switch FileFd to not transparently gunzip, since that breaks code which ↵martin@piware.de
expects the compressed contents to stay (such as the copy backend, or when using file:// repositories. Instead, introduce a new ReadOnlyGzip mode and use that where needed
2010-06-11apt-pkg/deb/debindexfile.cc: Fix one more place to check for gzipped ↵martin@piware.de
indexes, to work with apt-get source as well
2010-06-09* apt-pkg/deb/debindexfile.cc:martin@piware.de
- If we do not find uncompressed package/source/translation indexes, look for gzip compressed ones.