summaryrefslogtreecommitdiff
path: root/apt-pkg/deb/debindexfile.cc
AgeCommit message (Collapse)Author
2017-06-26fix some unlikely memory leaks in error casesDavid Kalnischkies
The error cases are just as unlikely as the memory leaks to ever cause real problems, but lets play it safe for correctness. Reported-By: scan-build & clang Gbp-Dch: Ignore
2016-08-26Add missing includes and external definitionsJulian Andres Klode
Several modules use std::array without including the array header. Bad modules. Some modules use STDOUT_FILENO and friends, or close() without including unistd.h, where they are defined. One module also uses WIFEXITED() without including sys/wait.h. Finally, environ is not specified to be defined in unistd.h. We are required to define it ourselves according to POSIX, so let's do that.
2016-08-17add --with-source option and Packages/Sources supportDavid Kalnischkies
We support "./foobar.deb" as a way to install a deb file directly. Recently .changes files were added. This highlights a problem as you can't add the changes file without also trying to install all of them. Now, it could also be handy to add entire Packages/Sources files to perhaps get a bunch of packages in without installing them all implicitly. This commit introduces --with-source which allows to add *.deb, *.changes, *.dsc, source-dirs, Packages & Sources files (the later can also be compressed) without also installing them.
2016-05-27prevent C++ locale number formatting in text APIsDavid Kalnischkies
Setting the C++ locale via std::locale::global(std::locale("")); which would otherwise default to the default C locale (aka: unaffected by setlocale) effects the formatting of numeric types in IO streams, which for output for humans is perfectly sensible, but breaks our many text interfaces used and parsed by us and others without expecting the numbers to be formatted. Closes: #825396
2016-04-03don't leak on error in listparser creationDavid Kalnischkies
Git-Dch: Ignore Reported-By: gcc -fsanitize=address
2016-01-25reimplement build-dep via apts normal resolverDavid Kalnischkies
build-dep was implemented by parsing the build-dependencies of a package and figuring out which packages to install/remove based on this. That means that for the first level of dependencies build-dep was implementing its very own resolver with all the benefits (aka: bugs) this gives us for not using the existing resolver for all levels. Making this work involves generating a dummy binary package with fitting Depends and Conflicts and as we can't create them out of thin air the cache generation needs to be involved so we end up writing a Packages file which we want to parse – after we have parsed the other Packages files already. With .dsc/.deb files we could add them before we started parsing anything. With a bit of care we can avoid generating too much data we have to throw away again (as many parts assume that e.g. the count of packages doesn't change midair), so that on a speed front there shouldn't be much of a difference, but output can be slightly confusing as if we have a completely valid cache on disk the "Reading package lists... Done" is printed two times – but apt is pretty quick about it in that case. Closes: #137560, #444930, #489911, #583914, #728317, #812173
2015-12-01require explicit paths to dsc/control as we do for deb filesDavid Kalnischkies
Otherwise a user is subject to unexpected content-injection depending on which directory she happens to start apt in. This also cleans up the code requiring less implementation details in build-dep which is always good. Technically, this is an ABI break as we override virtual methods, but that they weren't overridden was a mistake resulting in pure classes, which shouldn't be pure, so they were unusable – and as they are new in 1.1 nobody is using them yet (and hopefully ever as they are borderline implementation details). Closes: 806693
2015-11-04ignore newlines in dpkg-deb control output for installing debsDavid Kalnischkies
Leading or trailing newlines can be confusing for our parser as it expects two newlines to start/stop a new stanza. To solve this the lines we wanna add are printed first, ignore any leading newlines and then add the stanza as provided by dpkg-deb with or without trailing newlines as the parser will look at the first stanza only anyway and removing trailing newlines is considerably harder to do. Closes: 802553
2015-11-04show progress info while 'downloading' a local .deb fileDavid Kalnischkies
Showing just "Get: [1234 B]" looks very strange, so we now print the filename and as usual the package name, version and architecture.
2015-08-27Do not parse Status fields from remote sourcesJulian Andres Klode
This could allow an attacker to mark a package as installed in a remote package index, as long as the package was not listed in the dpkg status file. This way, an attacker could force the installation of a package during a dist-upgrade, by providing two packages in an index, an older marked as installed, and a newer - apt would "upgrade" to the newer version.
2015-08-17Fix all the wrong removals of includes that iwyu got wrongMichael Vogt
Git-Dch: ignore
2015-08-17Cleanup includes after running iwyuMichael Vogt
2015-08-10elimate duplicated code in pkgIndexFile subclassesDavid Kalnischkies
Trade deduplication of code for a bunch of new virtuals, so it is actually visible how the different indexes behave cleaning up the interface at large in the process. Git-Dch: Ignore
2015-08-10add volatile sources support in libapt-pkgDavid Kalnischkies
Sources are usually defined in sources.list (and co) and are pretty stable, but once in a while a frontend might want to add an additional "source" like a local .deb file to install this package (No support for 'real' sources being added this way as this is a multistep process). We had a hack in place to allow apt-get and apt to pull this of for a short while now, but other frontends are either left in the cold by this and/or the code for it looks dirty with FIXMEs plastering it and has on top of this also some problems (like including these 'volatile' sources in the srcpkgcache.bin file). So the biggest part in this commit is actually the rewrite of the cache generation as it is now potentially a three step process. The biggest problem with adding support now through is that this makes a bunch of previously mostly unusable by externs and therefore hidden classes public, so a bit of further tuneing on this now public API is in order…
2015-08-10add c++11 override marker to overridden methodsDavid Kalnischkies
C++11 adds the 'override' specifier to mark that a method is overriding a base class method and error out if not. We hide it in the APT_OVERRIDE macro to ensure that we keep compiling in pre-c++11 standards. Reported-By: clang-modernize -add-override -override-macros Git-Dch: Ignore
2015-08-10support lang= and target= sources.list optionsDavid Kalnischkies
We support arch= for a while, now we finally add lang= as well and as a first simple way of controlling which targets to acquire also target=. This asked for a redesign of the internal API of parsing and storing information about 'deb' and 'deb-src' lines. As this API isn't visible to the outside no damage done through. Beside being a nice cleanup (= it actually does more in less lines) it also provides us with a predictable order of architectures as provides in the configuration rather than based on string sorting-order, so that now the native architecture is parsed/displayed first. Observeable e.g. in apt-get output.
2015-08-10make all d-pointer * const pointersDavid Kalnischkies
Doing this disables the implicit copy assignment operator (among others) which would cause hovac if used on the classes as it would just copy the pointer, not the data the d-pointer points to. For most of the classes we don't need a copy assignment operator anyway and in many classes it was broken before as many contain a pointer of some sort. Only for our Cacheset Container interfaces we define an explicit copy assignment operator which could later be implemented to copy the data from one d-pointer to the other if we need it. Git-Dch: Ignore
2015-06-16add d-pointer, virtual destructors and de-inline de/constructorsDavid Kalnischkies
To have a chance to keep the ABI for a while we need all three to team up. One of them missing and we might loose, so ensuring that they are available is a very tedious but needed task once in a while. Git-Dch: Ignore
2015-06-15hide Translation-* in 'apt-cache policy' outputDavid Kalnischkies
Translation-* files are internally handled as PackageFiles which isn't super nice, but giving them their own struct is a bit overkill so let it be for the moment. They always appeared in the policy output because of this through and now that they are properly linked to a ReleaseFile they even display all the pinning information on them, but they don't contain any packages which could be pinned… No problem, but useless and potentially confusing output. Adding a 'NoPackages' flag which can be set on those files and be used in applications seems like a simple way to fix this display issue.
2015-06-15populate the Architecture field for PackageFilesDavid Kalnischkies
This is mainly visible in the policy, so that you can now pin by b= and let it only effect Packages files of this architecture and hence the packages coming from it (which do not need to be from this architecture, but very likely are in a normal repository setup). If you should pin by architecture in this way is a different question… Closes: 687255
2015-06-12store Release files data in the CacheDavid Kalnischkies
We used to read the Release file for each Packages file and store the data in the PackageFile struct even through potentially many Packages (and Translation-*) files could use the same data. The point of the exercise isn't the duplicated data through. Having the Release files as first-class citizens in the Cache allows us to properly track their state as well as allows us to use the information also for files which aren't in the cache, but where we know to which Release file they belong (Sources are an example for this). This modifies the pkgCache structs, especially the PackagesFile struct which depending on how libapt users access the data in these structs can mean huge breakage or no visible change. As a single data point: aptitude seems to be fine with this. Even if there is breakage it is trivial to fix in a backportable way while avoiding breakage for everyone would be a huge pain for us. Note that not all PackageFile structs have a corresponding ReleaseFile. In particular the dpkg/status file as well as *.deb files have not. As these have only a Archive property need, the Component property takes over this duty and the ReleaseFile remains zero. This is also the reason why it isn't needed nor particularily recommended to change from PackagesFile to ReleaseFile blindly. Sticking with the earlier is usually the better option.
2015-06-11use an enum instead of strings as IndexTarget::Option interfaceDavid Kalnischkies
Strings are easy to typo and we can keep the extensibility we require here with a simple enum we can append to without endangering ABI. Git-Dch: Ignore
2015-06-11use IndexTarget to get to IndexFileDavid Kalnischkies
Removes a bunch of duplicated code in the deb-specific parts. Especially the Description part is now handled centrally by IndexTarget instead of being duplicated to the derivations of IndexFile. Git-Dch: Ignore
2015-06-11show URI.Path in all acquire item descriptionsDavid Kalnischkies
It is a rather strange sight that index items use SiteOnly which strips the Path, while e.g. deb files are downloaded with NoUserPassword which does not. Important to note here is that for the file transport Path is pretty important as there is no Host which would be displayed by Site, which always resulted in "interesting" unspecific errors for "file:". Adding a 'middle' ground between the two which does show the Path but potentially modifies it (it strips a pending / at the end if existing) solves this "file:" issue, syncs the output and in the end helps to identify which file is meant exactly in progress output and co as a single site can have multiple repositories in different paths.
2015-06-09rework hashsum verification in the acquire systemDavid Kalnischkies
Having every item having its own code to verify the file(s) it handles is an errorprune process and easy to break, especially if items move through various stages (download, uncompress, patching, …). With a giant rework we centralize (most of) the verification to have a better enforcement rate and (hopefully) less chance for bugs, but it breaks the ABI bigtime in exchange – and as we break it anyway, it is broken even harder. It shouldn't effect most frontends as they don't deal with the acquire system at all or implement their own items, but some do and will need to be patched (might be an opportunity to use apt on-board material). The theory is simple: Items implement methods to decide if hashes need to be checked (in this stage) and to return the expected hashes for this item (in this stage). The verification itself is done in worker message passing which has the benefit that a hashsum error is now a proper error for the acquire system rather than a Done() which is later revised to a Failed().
2015-05-11stop depending on copy-on-write for std::stringDavid Kalnischkies
In 66c3875df391b1120b43831efcbe88a78569fbfe we workaround/fixed a problem where the code makes the assumption that the compiler uses copy-on-write implementations for std::string. Turns out that for c++11 compatibility gcc >= 5 will stop doing this by default.
2015-03-16properly implement pkgRecord::Parser for *.deb filesDavid Kalnischkies
Implementing FileName() works for most cases for us, but other frontends might need more and even for us its not very stable as the normal Jump() implementation is pretty bad on a deb file and produce errors on its own at times. So, replacing this makeshift with a complete implementation by mostly just shuffling code around.
2014-11-08mark internal interfaces as hiddenDavid Kalnischkies
We have a bunch of classes which are of no use for the outside world, but were still exported and so needed to preserve ABI/API. Marking them as hidden to not export them any longer is a big API break in theory, but in practice nobody is using them – as if they would its a bug.
2014-11-05apt-pkg/deb/debindexfile.{cc,h}: kill GetIndexes()Michael Vogt
2014-10-29Only support Translation-* that are listed in the {In,}Release fileMichael Vogt
Handle Translation-* files exactly like Packages files (with the expection that it is ok if a download of them fails). Remove all "guessing" on apts side. This will elimimnate a bunch of errors releated to captive portals and similar. Its also more correct and removes another potential attack vector.
2014-10-13do not inline virtual destructors with d-pointersDavid Kalnischkies
Reimplementing an inline method is opening a can of worms we don't want to open if we ever want to us a d-pointer in those classes, so we do the only thing which can save us from hell: move the destructors into the cc sources and we are good. Technically not an ABI break as the methods inline or not do the same (nothing), so a program compiled against the old version still works with the new version (beside that this version is still in experimental, so nothing really has been build against this library anyway). Git-Dch: Ignore
2014-09-30mark private methods as hiddenDavid Kalnischkies
We are the only possible users of private methods, so we are also the only users who can potentially export them via using them in inline methods. The point is: We don't need these symbols exported if we don't do this, so marking them as hidden removes some methods from the API without breaking anything as nobody could have used them. Git-Dch: Ignore
2014-09-29Merge branch 'debian/sid' into debian/experimentalMichael Vogt
Conflicts: apt-pkg/acquire-item.cc
2014-09-27drop stored StringItems in favor of in-memory mappingsDavid Kalnischkies
Strings like Section names or architectures are needed vary often. Instead of writing them each time we need them, we deploy sharing for these special strings. Until now, this was done with a linked list of strings in which we would search, which was stored in the cache. It turns out we can do this just as well in memory as well with a bunch of std::map's. In memory means here that it isn't available anymore if we have a partly invalid cache, but that isn't much of a problem in practice as the status file is compared to the other files we parse very small and includes mostly duplicates, so the space we would gain by storing is more or less equal to the size of the stored linked list…
2014-09-21generalize Acquire::GzipIndexMichael Vogt
2014-07-10Allow passing a full path to apt-get install /foo/bar.debMichael Vogt
CLoses: #752327
2014-06-18cleanup datatypes mix used in binary cacheDavid Kalnischkies
We had a wild mixture of (unsigned) int, long and long long here without much sense, so this commit adds a few typedefs to get some sense in the typesystem and ensures that a ID isn't sometimes computed as int, stored as long and compared with a long long… as this could potentially bite us later on as the size of the archive only increases over time.
2014-05-29Merge remote-tracking branch 'upstream/debian/sid' into debian/experimentalMichael Vogt
Conflicts: test/integration/test-bug-747261-arch-specific-conflicts
2014-05-15Never parse Version/Architecture tags in a Translation-$lang fileMichael Vogt
Version/Architecture information in a Translation-$lang file is not allowed, so don't try to parse it. This is a fix for a bugreport where a Translation-en file contained the content of the regular Packages file (probably due to local FS corruption). This lead to strange error messages on file download. Thanks to Thomas Reusch for the report.
2014-05-08apt-pkg/deb/debindexfile.cc: do not hardcode dpkgMichael Vogt
2014-05-08Merge remote-tracking branch 'mvo/feature/build-dep-dsc2' into ↵Michael Vogt
debian/experimental Conflicts: apt-pkg/deb/debindexfile.cc apt-pkg/deb/debindexfile.h apt-pkg/deb/debsrcrecords.cc
2014-04-28use flAbsPath()Michael Vogt
2014-04-28use new Popen()Michael Vogt
2014-04-28WIP local deb installMichael Vogt
2014-04-25add support for apt-get build-dep unpacked-source-dirMichael Vogt
2014-04-24create debIFTypeDscFile typeMichael Vogt
2014-03-13follow method attribute suggestions by gccDavid Kalnischkies
Git-Dch: Ignore Reported-By: gcc -Wsuggest-attribute={pure,const,noreturn}
2014-03-13cleanup headers and especially #includes everywhereDavid Kalnischkies
Beside being a bit cleaner it hopefully also resolves oddball problems I have with high levels of parallel jobs. Git-Dch: Ignore Reported-By: iwyu (include-what-you-use)
2013-03-18* apt-pkg/deb/debindexfile.cc,David Kalnischkies
apt-pkg/deb/deblistparser.cc: - use OpenMaybeClearSignedFile to be free from detecting and skipping clearsigning metadata in dsc and Release files We can't write a "clean" file to disk as not all acquire methods copy Release files before checking them (e.g. cdrom), so this reverts recombining, but uses the method we use for dsc files also in the two places we deal with Release files
2012-06-16do not dereference the storage for the unique strings as the pointer canDavid Kalnischkies
change at the time of writing the strings, so first store it temporary and then save the index in the (possibily new) pointer location