summaryrefslogtreecommitdiff
path: root/apt-pkg/deb/debmetaindex.cc
AgeCommit message (Collapse)Author
2015-06-12store Release files data in the CacheDavid Kalnischkies
We used to read the Release file for each Packages file and store the data in the PackageFile struct even through potentially many Packages (and Translation-*) files could use the same data. The point of the exercise isn't the duplicated data through. Having the Release files as first-class citizens in the Cache allows us to properly track their state as well as allows us to use the information also for files which aren't in the cache, but where we know to which Release file they belong (Sources are an example for this). This modifies the pkgCache structs, especially the PackagesFile struct which depending on how libapt users access the data in these structs can mean huge breakage or no visible change. As a single data point: aptitude seems to be fine with this. Even if there is breakage it is trivial to fix in a backportable way while avoiding breakage for everyone would be a huge pain for us. Note that not all PackageFile structs have a corresponding ReleaseFile. In particular the dpkg/status file as well as *.deb files have not. As these have only a Archive property need, the Component property takes over this duty and the ReleaseFile remains zero. This is also the reason why it isn't needed nor particularily recommended to change from PackagesFile to ReleaseFile blindly. Sticking with the earlier is usually the better option.
2015-06-11implement 'apt-get files' to access index targetsDavid Kalnischkies
Downloading additional files is only half the job. We still need a way to allow external tools to know where the files are they requested for download given that we don't want them to choose their own location. 'apt-get files' is our answer to this showing by default in a deb822 format information about each IndexTarget with the potential to filter the records based on lines and an option to change the output format. The command serves also as an example on how to get to this information via libapt.
2015-06-11use an enum instead of strings as IndexTarget::Option interfaceDavid Kalnischkies
Strings are easy to typo and we can keep the extensibility we require here with a simple enum we can append to without endangering ABI. Git-Dch: Ignore
2015-06-11use IndexTarget to get to IndexFileDavid Kalnischkies
Removes a bunch of duplicated code in the deb-specific parts. Especially the Description part is now handled centrally by IndexTarget instead of being duplicated to the derivations of IndexFile. Git-Dch: Ignore
2015-06-11show URI.Path in all acquire item descriptionsDavid Kalnischkies
It is a rather strange sight that index items use SiteOnly which strips the Path, while e.g. deb files are downloaded with NoUserPassword which does not. Important to note here is that for the file transport Path is pretty important as there is no Host which would be displayed by Site, which always resulted in "interesting" unspecific errors for "file:". Adding a 'middle' ground between the two which does show the Path but potentially modifies it (it strips a pending / at the end if existing) solves this "file:" issue, syncs the output and in the end helps to identify which file is meant exactly in progress output and co as a single site can have multiple repositories in different paths.
2015-06-10rename Calculate- to GetIndexTargets and use it as official APIDavid Kalnischkies
We need a general way to get from a sources.list entry to IndexTargets and with this change we can move from pkgSourceList over the list of metaIndexes it includes to the IndexTargets each metaIndex can have. Git-Dch: Ignore
2015-06-10stop using IndexTarget pointers which are never freedDavid Kalnischkies
Creating and passing around a bunch of pointers of IndexTargets (and of a vector of pointers of IndexTargets) is probably done to avoid the 'costly' copy of container, but we are really not in a timecritical operation here and move semantics will help us even further in the future. On the other hand we never do a proper cleanup of these pointers, which is very dirty, even if structures aren't that big… The changes will effecting many items only effect our own hidden class, so we can do that without fearing breaking interfaces or anything. Git-Dch: Ignore
2015-06-10store all targets data in IndexTarget structDavid Kalnischkies
We still need an API for the targets, so slowly prepare the IndexTargets to let them take this job. Git-Dch: Ignore
2015-06-10abstract the code to iterate over all targets a bitDavid Kalnischkies
We have two places in the code which need to iterate over targets and do certain things with it. The first one is actually creating these targets for download and the second instance pepares certain targets for reading. Git-Dch: Ignore
2015-06-09configureable acquire targets to download additional filesDavid Kalnischkies
First pass at making the acquire system capable of downloading files based on configuration rather than hardcoded entries. It is now possible to instruct 'deb' and 'deb-src' sources.list lines to download more than just Packages/Translation-* and Sources files. Details on how to do that can be found in the included documentation file.
2015-06-09rework hashsum verification in the acquire systemDavid Kalnischkies
Having every item having its own code to verify the file(s) it handles is an errorprune process and easy to break, especially if items move through various stages (download, uncompress, patching, …). With a giant rework we centralize (most of) the verification to have a better enforcement rate and (hopefully) less chance for bugs, but it breaks the ABI bigtime in exchange – and as we break it anyway, it is broken even harder. It shouldn't effect most frontends as they don't deal with the acquire system at all or implement their own items, but some do and will need to be patched (might be an opportunity to use apt on-board material). The theory is simple: Items implement methods to decide if hashes need to be checked (in this stage) and to return the expected hashes for this item (in this stage). The verification itself is done in worker message passing which has the benefit that a hashsum error is now a proper error for the acquire system rather than a Done() which is later revised to a Failed().
2015-03-16fix some new compiler warnings reported by gcc-5David Kalnischkies
Git-Dch: Ignore
2014-11-08mark internal interfaces as hiddenDavid Kalnischkies
We have a bunch of classes which are of no use for the outside world, but were still exported and so needed to preserve ABI/API. Marking them as hidden to not export them any longer is a big API break in theory, but in practice nobody is using them – as if they would its a bug.
2014-11-08better non-virtual metaIndex.LocalFileName() implementationDavid Kalnischkies
We can't add a new virtual method without breaking the ABI, but we can freely add new methods, so for older ABIs we just implement this method with a dynamic_cast, so that clients can be more ignorant about the API here and especially don't need to pull a very dirty trick by assuming internal knowledge (like apt-get did here).
2014-11-08use a abi version check similar to the gcc checkDavid Kalnischkies
Git-Dch: Ignore
2014-10-13trusted=yes sources are secure, we just don't know whyDavid Kalnischkies
Do not require a special flag to be present to update trusted=yes sources as this flag in the sources.list is obviously special enough. Note that this is just disabling the error message, the user will still be warned about all the (possible) failures the repository generated, it is just triggering the acceptance of the warnings on a source-by-source level. Similarily, the trusted=no flag doesn't require the user to pass additional flags to update, if the repository looks fine in the view of apt it will update just fine. The unauthenticated warnings will "just" be presented then the data is used. In case you wonder: Both was the behavior in previous versions, too.
2014-09-17use pkgAcqMetaBase as the transactionManagerMichael Vogt
2014-07-31Rework TransactionID stuffMichael Vogt
2014-07-21Download Release first, then Release.gpgMichael Vogt
The old way of handling this was that pkgAcqMetaIndex was responsible to check/move both Release and Release.gpg in place. This breaks the assumption of the transaction that each pkgAcquire::Item has a single File that its responsible for.
2014-05-29Fix more warnings from clangMichael Vogt
Reported-By: clang++ -Werror Conflicts: apt-pkg/acquire-item.cc apt-pkg/acquire-item.h apt-pkg/deb/debmetaindex.h
2014-05-09use HashStringList in the acquire systemDavid Kalnischkies
It is not very extensible to have the supported Hashes hardcoded everywhere and especially if it is part of virtual method names. It is also possible that a method does not support the 'best' hash (yet), so we might end up not being able to verify a file even though we have a common subset of supported hashes. And those are just two of the cases in which it is handy to have a more dynamic selection. The downside is that this is a MAJOR API break, but the HashStringList has a string constructor for compatibility, so with a bit of luck the few frontends playing with the acquire system directly are okay.
2014-05-08Merge remote-tracking branch 'mvo/feature/build-dep-dsc2' into ↵Michael Vogt
debian/experimental Conflicts: apt-pkg/deb/debindexfile.cc apt-pkg/deb/debindexfile.h apt-pkg/deb/debsrcrecords.cc
2014-04-28avoid deb specific code in private-installMichael Vogt
2014-04-28WIP local deb installMichael Vogt
2014-03-13cleanup headers and especially #includes everywhereDavid Kalnischkies
Beside being a bit cleaner it hopefully also resolves oddball problems I have with high levels of parallel jobs. Git-Dch: Ignore Reported-By: iwyu (include-what-you-use)
2013-12-05* enable release based selection for deb-src (closes: 731102)Michael Vogt
2013-07-25delete targets dataMichael Vogt
2013-07-11implement arch+= and arch-= for sources.listDavid Kalnischkies
Default is to acquire all architectures from APT::Architectures which can be changed by arch=, but this isn't very flexible if you want "mostly" the default as you have to hardcode the architectures then, so arch-= and arch+= can be used to add/remove architectures from the default set. On a machine with 'amd64' and 'i386' configured the lines: deb [arch+=armel] http://example.org/debian wheezy rocks deb [arch-=amd64] http://example.org/debian jessie rocks will result in the download of: wheezy Packages for 'amd64', 'i386' and 'armel' jessie Packages for 'i386'
2013-03-15 - if ExecGPGV deals with a clear-signed file it will split this fileDavid Kalnischkies
into data and signatures, pass it to gpgv for verification and recombines it after that in a known-good way without unsigned blocks and whitespaces resulting usually in more or less the same file as before, but later code can be sure about the format * apt-pkg/deb/debmetaindex.cc: - reenable InRelease by default
2013-03-14* SECURITY UPDATE: InRelease verification bypass0.9.7.8Michael Vogt
- CVE-2013-1051 * apt-pkg/deb/debmetaindex.cc, test/integration/test-bug-595691-empty-and-broken-archive-files, test/integration/test-releasefile-verification: - disable InRelease downloading until the verification issue is fixed, thanks to Ansgar Burchardt for finding the flaw
2012-02-18* apt-pkg/acquire-item.cc:David Kalnischkies
- drop support for i18n/Index file (introduced in 0.8.11) and use the Release file instead to get the Translations (Closes: #649314) * ftparchive/writer.cc: - add 'Translation-*' to the default patterns i18n/Index was never used outside debian - and even here it isn't used consistently as only 'main' has such a file. As the Release file now includes the Translation-* files we therefore drop support for i18n/Index. A version supporting it was never part of a debian release and still supporting it would mean that we get 99% of the time a 404 as response to the request anyway and confuse archive maintainers who want to provide all files APT tries to acquire.
2011-10-05* apt-pkg/deb/debmetaindex.cc:David Kalnischkies
- none is a separator, not a language: no need for Index (Closes: #624218) * apt-pkg/aptconfiguration.cc: - do not builtin languages only if none is forced (Closes: #643787)
2011-10-05merge with current debian apt/experimentalDavid Kalnischkies
2011-10-05* apt-pkg/contrib/configuration.cc:Michael Vogt
- fix double delete (LP: #848907) - ignore only the invalid regexp instead of all options * apt-pkg/acquire-item.h, apt-pkg/deb/debmetaindex.cc: - fix fetching language information by adding OptionalSubIndexTarget * methods/https.cc: - cleanup broken downloads properly * ftparchive/cachedb.cc: - fix buffersize in bytes2hex * apt-pkg/deb/deblistparser.cc: - fix crash when the dynamic mmap needs to be grown in LoadReleaseInfo (LP: #854090)
2011-09-19use forward declaration in headers if possible instead of includesDavid Kalnischkies
2011-09-16* apt-pkg/acquire-item.h, apt-pkg/deb/debmetaindex.cc:Michael Vogt
- fix fetching language information by adding OptionalSubIndexTarget
2011-09-13merge with debian/experimentalDavid Kalnischkies
2011-09-13reorder includes: add <config.h> if needed and include it at firstDavid Kalnischkies
2011-08-15merged from the debian-sid branchMichael Vogt
2011-08-11cppcheck complains about some possible speed improvements which could beDavid Kalnischkies
done on the mirco-optimazation level, so lets fix them: (performance) Possible inefficient checking for emptiness. (performance) Prefer prefix ++/-- operators for non-primitive types.
2011-07-14* apt-pkg/deb/debmetaindex.cc:David Kalnischkies
- add trusted=yes option to mark unsigned (local) repository as trusted based on a patch from Ansgar Burchardt, thanks a lot! (Closes: #596498) Note that "apt-get update" still warns about unknown signatures even when [trusted=yes] is given for the source.
2011-04-26* apt-pkg/deb/debmetaindex.cc:David Kalnischkies
- do not download TranslationIndex if no Translation-* will be downloaded later on anyway (Closes: #624218)
2011-01-28 - download and use i18n/Index to choose which Translations to downloadDavid Kalnischkies
* apt-pkg/aptconfiguration.cc: - remove the inbuilt Translation files whitelist
2011-01-20 - try downloading clearsigned InRelease before trying Release.gpgDavid Kalnischkies
* apt-pkg/deb/deblistparser.cc: - rewrite LoadReleaseInfo to cope with clearsigned Releasefiles
2010-07-11fix typo preventing display of architecture in Info()David Kalnischkies
2010-07-11* apt-pkg/deb/debmetaindex.cc:David Kalnischkies
- do not query each architecture for flat file archives
2010-03-30rename ExplodeString to VectorizeStringDavid Kalnischkies
2010-02-18doesn't use a default separator in ExplodeString (halfway losted in merge)David Kalnischkies
2010-02-15* aptconfiguration.cc:David Kalnischkies
- include all existing Translation files in the Cache (Closes: 564137) Previously if APT was executed with a different LC_* all these invocations needed to rebuild the Cache as too many files were included or missing: Now the lists-directory is checked for Translation-files and all these included in getLanguages() regardless of the environment setting (after a "none" so APT will not use them for displaying information).
2009-12-23merge Goswin Brederlow "support download of index files for different archs"David Kalnischkies
patch which includes the following big changes: - Declare the unused [vendor] field in sources.list as option field, e.g. deb [arch=amd64,i386 lang=en_GB have=fun] http://example.org - When fetching index files download them for all APT::Architectures (overrideable with the options field above) - Allow all architectures of APT::Architectures to be in the Cache - Add the architecture to status and progress informations - Add b= (Binary architecture) to policy This commit doesn't incude the "pin-hack" as the Group structure will take care of this (and does it already to some extend).