summaryrefslogtreecommitdiff
path: root/apt-pkg/deb/debmetaindex.cc
AgeCommit message (Collapse)Author
2015-08-10add sources.list Check-Valid-Until and Valid-Until-{Max,Min} optionsDavid Kalnischkies
These options could be set via configuration before, but the connection to the actual sources is so strong that they should really be set in the sources.list instead – especially as this can be done a lot more specific rather than e.g. disabling Valid-Until for all sources at once. Valid-Until-* names are chosen instead of the Min/Max-ValidTime as this seems like a better name and their use in the wild is probably low enough that this isn't going to confuse anyone if we have to names for the same thing in different areas. In the longrun, the config options should be removed, but for now documentation hinting at the new options is good enough as these are the kind of options you set once across many systems with different apt versions, so the new way should work everywhere first before we deprecate the old way.
2015-08-10merge indexRecords into metaIndexDavid Kalnischkies
indexRecords was used to parse the Release file – mostly the hashes – while metaIndex deals with downloading the Release file, storing all indexes coming from this release and … parsing the Release file, but this time mostly for the other fields. That wasn't a problem in metaIndex as this was done in the type specific subclass, but indexRecords while allowing to override the parsing method did expect by default a specific format. APT isn't really supporting different types at the moment, but this is a violation of the abstraction we have everywhere else and, which is the actual reason for this merge: Options e.g. coming from the sources.list come to metaIndex naturally, which needs to wrap them up and bring them into indexRecords, so the acquire system is told about it as they don't get to see the metaIndex, but they don't really belong in indexRecords as this is just for storing data loaded from the Release file… the result is a complete mess. I am not saying it is a lot prettier after the merge, but at least adding new options is now slightly easier and there is just one place responsible for parsing the Release file. That can't hurt.
2015-08-10detect and error out on conflicting Trusted settingsDavid Kalnischkies
A specific trust state can be enforced via a sources.list option, but it effects all entries handled by the same Release file, not just the entry it was given on so we enforce acknowledgement of this by requiring the same value to be (not) set on all such entries.
2015-08-10support lang= and target= sources.list optionsDavid Kalnischkies
We support arch= for a while, now we finally add lang= as well and as a first simple way of controlling which targets to acquire also target=. This asked for a redesign of the internal API of parsing and storing information about 'deb' and 'deb-src' lines. As this API isn't visible to the outside no damage done through. Beside being a nice cleanup (= it actually does more in less lines) it also provides us with a predictable order of architectures as provides in the configuration rather than based on string sorting-order, so that now the native architecture is parsed/displayed first. Observeable e.g. in apt-get output.
2015-08-10fix memory leaks reported by -fsanitizeDavid Kalnischkies
Various small leaks here and there. Nothing particularily big, but still good to fix. Found by the sanitizers while running our testcases. Reported-By: gcc -fsanitize Git-Dch: Ignore
2015-08-10make all d-pointer * const pointersDavid Kalnischkies
Doing this disables the implicit copy assignment operator (among others) which would cause hovac if used on the classes as it would just copy the pointer, not the data the d-pointer points to. For most of the classes we don't need a copy assignment operator anyway and in many classes it was broken before as many contain a pointer of some sort. Only for our Cacheset Container interfaces we define an explicit copy assignment operator which could later be implemented to copy the data from one d-pointer to the other if we need it. Git-Dch: Ignore
2015-08-10apply various style suggestions by cppcheckDavid Kalnischkies
Some of them modify the ABI, but given that we prepare a big one already, these few hardly count for much. Git-Dch: Ignore
2015-06-16add d-pointer, virtual destructors and de-inline de/constructorsDavid Kalnischkies
To have a chance to keep the ABI for a while we need all three to team up. One of them missing and we might loose, so ensuring that they are available is a very tedious but needed task once in a while. Git-Dch: Ignore
2015-06-15implement default apt-get file --release-info modeDavid Kalnischkies
Selecting targets based on the Release they belong to isn't to unrealistic. In fact, it is assumed to be the most used case so it is made the default especially as this allows to bundle another thing we have to be careful with: Filenames and only showing targets we have acquired. Closes: 752702
2015-06-12store Release files data in the CacheDavid Kalnischkies
We used to read the Release file for each Packages file and store the data in the PackageFile struct even through potentially many Packages (and Translation-*) files could use the same data. The point of the exercise isn't the duplicated data through. Having the Release files as first-class citizens in the Cache allows us to properly track their state as well as allows us to use the information also for files which aren't in the cache, but where we know to which Release file they belong (Sources are an example for this). This modifies the pkgCache structs, especially the PackagesFile struct which depending on how libapt users access the data in these structs can mean huge breakage or no visible change. As a single data point: aptitude seems to be fine with this. Even if there is breakage it is trivial to fix in a backportable way while avoiding breakage for everyone would be a huge pain for us. Note that not all PackageFile structs have a corresponding ReleaseFile. In particular the dpkg/status file as well as *.deb files have not. As these have only a Archive property need, the Component property takes over this duty and the ReleaseFile remains zero. This is also the reason why it isn't needed nor particularily recommended to change from PackagesFile to ReleaseFile blindly. Sticking with the earlier is usually the better option.
2015-06-11implement 'apt-get files' to access index targetsDavid Kalnischkies
Downloading additional files is only half the job. We still need a way to allow external tools to know where the files are they requested for download given that we don't want them to choose their own location. 'apt-get files' is our answer to this showing by default in a deb822 format information about each IndexTarget with the potential to filter the records based on lines and an option to change the output format. The command serves also as an example on how to get to this information via libapt.
2015-06-11use an enum instead of strings as IndexTarget::Option interfaceDavid Kalnischkies
Strings are easy to typo and we can keep the extensibility we require here with a simple enum we can append to without endangering ABI. Git-Dch: Ignore
2015-06-11use IndexTarget to get to IndexFileDavid Kalnischkies
Removes a bunch of duplicated code in the deb-specific parts. Especially the Description part is now handled centrally by IndexTarget instead of being duplicated to the derivations of IndexFile. Git-Dch: Ignore
2015-06-11show URI.Path in all acquire item descriptionsDavid Kalnischkies
It is a rather strange sight that index items use SiteOnly which strips the Path, while e.g. deb files are downloaded with NoUserPassword which does not. Important to note here is that for the file transport Path is pretty important as there is no Host which would be displayed by Site, which always resulted in "interesting" unspecific errors for "file:". Adding a 'middle' ground between the two which does show the Path but potentially modifies it (it strips a pending / at the end if existing) solves this "file:" issue, syncs the output and in the end helps to identify which file is meant exactly in progress output and co as a single site can have multiple repositories in different paths.
2015-06-10rename Calculate- to GetIndexTargets and use it as official APIDavid Kalnischkies
We need a general way to get from a sources.list entry to IndexTargets and with this change we can move from pkgSourceList over the list of metaIndexes it includes to the IndexTargets each metaIndex can have. Git-Dch: Ignore
2015-06-10stop using IndexTarget pointers which are never freedDavid Kalnischkies
Creating and passing around a bunch of pointers of IndexTargets (and of a vector of pointers of IndexTargets) is probably done to avoid the 'costly' copy of container, but we are really not in a timecritical operation here and move semantics will help us even further in the future. On the other hand we never do a proper cleanup of these pointers, which is very dirty, even if structures aren't that big… The changes will effecting many items only effect our own hidden class, so we can do that without fearing breaking interfaces or anything. Git-Dch: Ignore
2015-06-10store all targets data in IndexTarget structDavid Kalnischkies
We still need an API for the targets, so slowly prepare the IndexTargets to let them take this job. Git-Dch: Ignore
2015-06-10abstract the code to iterate over all targets a bitDavid Kalnischkies
We have two places in the code which need to iterate over targets and do certain things with it. The first one is actually creating these targets for download and the second instance pepares certain targets for reading. Git-Dch: Ignore
2015-06-09configureable acquire targets to download additional filesDavid Kalnischkies
First pass at making the acquire system capable of downloading files based on configuration rather than hardcoded entries. It is now possible to instruct 'deb' and 'deb-src' sources.list lines to download more than just Packages/Translation-* and Sources files. Details on how to do that can be found in the included documentation file.
2015-06-09rework hashsum verification in the acquire systemDavid Kalnischkies
Having every item having its own code to verify the file(s) it handles is an errorprune process and easy to break, especially if items move through various stages (download, uncompress, patching, …). With a giant rework we centralize (most of) the verification to have a better enforcement rate and (hopefully) less chance for bugs, but it breaks the ABI bigtime in exchange – and as we break it anyway, it is broken even harder. It shouldn't effect most frontends as they don't deal with the acquire system at all or implement their own items, but some do and will need to be patched (might be an opportunity to use apt on-board material). The theory is simple: Items implement methods to decide if hashes need to be checked (in this stage) and to return the expected hashes for this item (in this stage). The verification itself is done in worker message passing which has the benefit that a hashsum error is now a proper error for the acquire system rather than a Done() which is later revised to a Failed().
2015-03-16fix some new compiler warnings reported by gcc-5David Kalnischkies
Git-Dch: Ignore
2014-11-08mark internal interfaces as hiddenDavid Kalnischkies
We have a bunch of classes which are of no use for the outside world, but were still exported and so needed to preserve ABI/API. Marking them as hidden to not export them any longer is a big API break in theory, but in practice nobody is using them – as if they would its a bug.
2014-11-08better non-virtual metaIndex.LocalFileName() implementationDavid Kalnischkies
We can't add a new virtual method without breaking the ABI, but we can freely add new methods, so for older ABIs we just implement this method with a dynamic_cast, so that clients can be more ignorant about the API here and especially don't need to pull a very dirty trick by assuming internal knowledge (like apt-get did here).
2014-11-08use a abi version check similar to the gcc checkDavid Kalnischkies
Git-Dch: Ignore
2014-10-13trusted=yes sources are secure, we just don't know whyDavid Kalnischkies
Do not require a special flag to be present to update trusted=yes sources as this flag in the sources.list is obviously special enough. Note that this is just disabling the error message, the user will still be warned about all the (possible) failures the repository generated, it is just triggering the acceptance of the warnings on a source-by-source level. Similarily, the trusted=no flag doesn't require the user to pass additional flags to update, if the repository looks fine in the view of apt it will update just fine. The unauthenticated warnings will "just" be presented then the data is used. In case you wonder: Both was the behavior in previous versions, too.
2014-09-17use pkgAcqMetaBase as the transactionManagerMichael Vogt
2014-07-31Rework TransactionID stuffMichael Vogt
2014-07-21Download Release first, then Release.gpgMichael Vogt
The old way of handling this was that pkgAcqMetaIndex was responsible to check/move both Release and Release.gpg in place. This breaks the assumption of the transaction that each pkgAcquire::Item has a single File that its responsible for.
2014-05-29Fix more warnings from clangMichael Vogt
Reported-By: clang++ -Werror Conflicts: apt-pkg/acquire-item.cc apt-pkg/acquire-item.h apt-pkg/deb/debmetaindex.h
2014-05-09use HashStringList in the acquire systemDavid Kalnischkies
It is not very extensible to have the supported Hashes hardcoded everywhere and especially if it is part of virtual method names. It is also possible that a method does not support the 'best' hash (yet), so we might end up not being able to verify a file even though we have a common subset of supported hashes. And those are just two of the cases in which it is handy to have a more dynamic selection. The downside is that this is a MAJOR API break, but the HashStringList has a string constructor for compatibility, so with a bit of luck the few frontends playing with the acquire system directly are okay.
2014-05-08Merge remote-tracking branch 'mvo/feature/build-dep-dsc2' into ↵Michael Vogt
debian/experimental Conflicts: apt-pkg/deb/debindexfile.cc apt-pkg/deb/debindexfile.h apt-pkg/deb/debsrcrecords.cc
2014-04-28avoid deb specific code in private-installMichael Vogt
2014-04-28WIP local deb installMichael Vogt
2014-03-13cleanup headers and especially #includes everywhereDavid Kalnischkies
Beside being a bit cleaner it hopefully also resolves oddball problems I have with high levels of parallel jobs. Git-Dch: Ignore Reported-By: iwyu (include-what-you-use)
2013-12-05* enable release based selection for deb-src (closes: 731102)Michael Vogt
2013-07-25delete targets dataMichael Vogt
2013-07-11implement arch+= and arch-= for sources.listDavid Kalnischkies
Default is to acquire all architectures from APT::Architectures which can be changed by arch=, but this isn't very flexible if you want "mostly" the default as you have to hardcode the architectures then, so arch-= and arch+= can be used to add/remove architectures from the default set. On a machine with 'amd64' and 'i386' configured the lines: deb [arch+=armel] http://example.org/debian wheezy rocks deb [arch-=amd64] http://example.org/debian jessie rocks will result in the download of: wheezy Packages for 'amd64', 'i386' and 'armel' jessie Packages for 'i386'
2013-03-15 - if ExecGPGV deals with a clear-signed file it will split this fileDavid Kalnischkies
into data and signatures, pass it to gpgv for verification and recombines it after that in a known-good way without unsigned blocks and whitespaces resulting usually in more or less the same file as before, but later code can be sure about the format * apt-pkg/deb/debmetaindex.cc: - reenable InRelease by default
2013-03-14* SECURITY UPDATE: InRelease verification bypass0.9.7.8Michael Vogt
- CVE-2013-1051 * apt-pkg/deb/debmetaindex.cc, test/integration/test-bug-595691-empty-and-broken-archive-files, test/integration/test-releasefile-verification: - disable InRelease downloading until the verification issue is fixed, thanks to Ansgar Burchardt for finding the flaw
2012-02-18* apt-pkg/acquire-item.cc:David Kalnischkies
- drop support for i18n/Index file (introduced in 0.8.11) and use the Release file instead to get the Translations (Closes: #649314) * ftparchive/writer.cc: - add 'Translation-*' to the default patterns i18n/Index was never used outside debian - and even here it isn't used consistently as only 'main' has such a file. As the Release file now includes the Translation-* files we therefore drop support for i18n/Index. A version supporting it was never part of a debian release and still supporting it would mean that we get 99% of the time a 404 as response to the request anyway and confuse archive maintainers who want to provide all files APT tries to acquire.
2011-10-05* apt-pkg/deb/debmetaindex.cc:David Kalnischkies
- none is a separator, not a language: no need for Index (Closes: #624218) * apt-pkg/aptconfiguration.cc: - do not builtin languages only if none is forced (Closes: #643787)
2011-10-05merge with current debian apt/experimentalDavid Kalnischkies
2011-10-05* apt-pkg/contrib/configuration.cc:Michael Vogt
- fix double delete (LP: #848907) - ignore only the invalid regexp instead of all options * apt-pkg/acquire-item.h, apt-pkg/deb/debmetaindex.cc: - fix fetching language information by adding OptionalSubIndexTarget * methods/https.cc: - cleanup broken downloads properly * ftparchive/cachedb.cc: - fix buffersize in bytes2hex * apt-pkg/deb/deblistparser.cc: - fix crash when the dynamic mmap needs to be grown in LoadReleaseInfo (LP: #854090)
2011-09-19use forward declaration in headers if possible instead of includesDavid Kalnischkies
2011-09-16* apt-pkg/acquire-item.h, apt-pkg/deb/debmetaindex.cc:Michael Vogt
- fix fetching language information by adding OptionalSubIndexTarget
2011-09-13merge with debian/experimentalDavid Kalnischkies
2011-09-13reorder includes: add <config.h> if needed and include it at firstDavid Kalnischkies
2011-08-15merged from the debian-sid branchMichael Vogt
2011-08-11cppcheck complains about some possible speed improvements which could beDavid Kalnischkies
done on the mirco-optimazation level, so lets fix them: (performance) Possible inefficient checking for emptiness. (performance) Prefer prefix ++/-- operators for non-primitive types.
2011-07-14* apt-pkg/deb/debmetaindex.cc:David Kalnischkies
- add trusted=yes option to mark unsigned (local) repository as trusted based on a patch from Ansgar Burchardt, thanks a lot! (Closes: #596498) Note that "apt-get update" still warns about unknown signatures even when [trusted=yes] is given for the source.