summaryrefslogtreecommitdiff
path: root/ftparchive
AgeCommit message (Collapse)Author
2015-11-04wrap every unlink call to check for != /dev/nullDavid Kalnischkies
Unlinking /dev/null is bad, we shouldn't do that. Also, we should print at least a warning if we tried to unlink a file but didn't manage to pull it of (ignoring the case were the file is /dev/null or doesn't exist in the first place). This got triggered by a relatively unlikely to cause problem in pkgAcquire::Worker::PrepareFiles which would while temporary uncompressed files (which are set to keep compressed) figure out that to files are the same and prepare for sharing by deleting them. Bad move. That also shows why not printing a warning is a bad idea as this hide the error for in non-root test runs. Git-Dch: Ignore
2015-11-04support arch:all data e.g. in separate Packages fileDavid Kalnischkies
Based on a discussion with Niels Thykier who asked for Contents-all this implements apt trying for all architecture dependent files to get a file for the architecture all, which is treated internally now as an official architecture which is always around (like native). This way arch:all data can be shared instead of duplicated for each architecture requiring the user to download the same information again and again. There is one problem however: In Debian there is already a binary-all/ Packages file, but the binary-any files still include arch:all packages, so that downloading this file now would be a waste of time, bandwidth and diskspace. We therefore need a way to decide if it makes sense to download the all file for Packages in Debian or not. The obvious answer would be a special flag in the Release file indicating this, which would need to default to 'no' and every reasonable repository would override it to 'yes' in a few years time, but the flag would be there "forever". Looking closer at a Release file we see the field "Architectures", which doesn't include 'all' at the moment. With the idea outlined above that 'all' is a "proper" architecture now, we interpret this field as being authoritative in declaring which architectures are supported by this repository. If it says 'all', apt will try to get all, if not it will be skipped. This gives us another interesting feature: If I configure a source to download armel and mips, but it declares it supports only armel apt will now print a notice saying as much. Previously this was a very cryptic failure. If on the other hand the repository supports mips, too, but for some reason doesn't ship mips packages at the moment, this 'missing' file is silently ignored (= that is the same as the repository including an empty file). The Architectures field isn't mandatory through, so if it isn't there, we assume that every architecture is supported by this repository, which skips the arch:all if not listed in the release file.
2015-11-04reenable gcc warnings for deprecated functionsDavid Kalnischkies
In the meantime the strange warnings disappeared, so we can get back to showing them – and fix the one occurance which creeped in in the meantime. Git-Dch: Ignore
2015-11-04revamp all tools help messagesDavid Kalnischkies
The general idea is: A small paragraph on the tool itself as a description, a list of the most used (!= all) commands available in the tool, a remark where to find more information on the tool and its commands (aka: in the manpage) and finally a common block referring to even more manpages. In exchange options are completely omitted from the output as well as deprecated or obscure commands. (Better) Information about them is available in the manpages anyway and the few options which were listed before were also the least interesting ones (-o -c -q and co are hardly of interest for someone totally new looking to find info by asking for help and anyone with a bit of experience doesn't need this short list. Those would need a list of options applying to the command they call, but they are too numerous and command specific to list them sanely in this context.
2015-11-04deal with --version more centrallyDavid Kalnischkies
Git-Dch: Ignore
2015-11-04move apts cmdline helper type into -privateDavid Kalnischkies
Its not as simple as I initially thought to abstract this enough to make it globally usable, so lets not pollute global namespace with this for now. Git-Dch: Ignore
2015-11-04generate commands array after config is loadedDavid Kalnischkies
This ensures that location strings loaded from a location specified via configuration (Dir::Locale) effect the help messages for commands. Git-Dch: Ignore
2015-11-04deduplicate main methodsDavid Kalnischkies
All mains pretty much do the same thing, so lets try a little harder to move the common parts into -private to have the real differences more visible. Git-Dch: Ignore
2015-11-04split up help messages for simpler reuseDavid Kalnischkies
That is one huge commit with busy work only: Help messages used to be one big translateable string, which is a pain for translators and hard to reuse for us. This change there 'explodes' this single string into new string for each documented string trying hard to split up the translated messages as well. This actually restores many translations as previously adding a single command made all of the bug message fuzzy. The splitup also highlighted that its easy to forget a line, duplicate one and similar stuff. Git-Dch: Ignore
2015-09-14add --sha512 option + documentation for apt-ftparchiveDavid Kalnischkies
2015-09-14use APT::FTPArchive hash settings as default for APT::FPArchive::$filetypeDavid Kalnischkies
2015-09-14do not generate bogus hashes if hash is disabled in apt-ftparchiveDavid Kalnischkies
2015-09-07Set APT::FTPArchive::DoByHash to false by defaultMichael Vogt
"apt-ftparchive release" will create the by-hash files if this mode is enabled. This maybe unexpected by existing users so make it a opt-in. Git-Dch: ignore
2015-09-07Ensure that no by-hash file for "Release/InRelease is written"Michael Vogt
Git-Dch: ignore Brown-paperbag: yes Thanks: Donkult
2015-09-04Add support for writing by-hash dirs in apt-ftparchiveMichael Vogt
This option is enabled via the APT::FTPArchive::DoByHash switch. It will also honor the option APT::FTPArchive::By-Hash-Keep that controls how many previous generation of by-hash files should be kept (defaults to 3). Merged from https://github.com/mvo5/apt/tree/feature/apt-ftparchive-by-hash
2015-08-14Replace UINT_MAX with std::numeric_limits<unsigned int>::max()Julian Andres Klode
Gbp-Dch: ignore
2015-08-13C++11: Switch from auto_ptr to unique_ptrJulian Andres Klode
This is nicer
2015-08-10add c++11 override marker to overridden methodsDavid Kalnischkies
C++11 adds the 'override' specifier to mark that a method is overriding a base class method and error out if not. We hide it in the APT_OVERRIDE macro to ensure that we keep compiling in pre-c++11 standards. Reported-By: clang-modernize -add-override -override-macros Git-Dch: Ignore
2015-08-10fix memory leaks reported by -fsanitizeDavid Kalnischkies
Various small leaks here and there. Nothing particularily big, but still good to fix. Found by the sanitizers while running our testcases. Reported-By: gcc -fsanitize Git-Dch: Ignore
2015-05-11rewrite all TFRewrite instances to use the new pkgTagSection::WriteDavid Kalnischkies
While it is mostly busywork to rewrite all instances it actually fixes bugs as the data storage used by the new method is std::string rather than a char*, the later mostly created by c_str() from a std::string which the caller has to ensure keeps in scope – something apt-ftparchive actually didn't ensure and relied on copy-on-write behavior instead which c++11 forbids and hence the new default gcc abi doesn't use it.
2015-05-11fix 'Source' to 'Package' rename in apt-ftparchiveDavid Kalnischkies
This rename with value is ordered by the 'old' name 'Source', but should be ordered by the new name… by splitting the operation in a delete and a new field we can easily fix this problem locally for now.
2015-04-19calculate only expected hashes in methodsDavid Kalnischkies
Methods get told which hashes are expected by the acquire system, which means we can use this list to restrict what we calculate in the methods as any extra we are calculating is wasted effort as we can't compare it with anything anyway. Adding support for a new hash algorithm is therefore 'free' now and if a algorithm is no longer provided in a repository for a file, we automatically stop calculating it. In practice this results in a speed-up in Debian as we don't have SHA512 here (so far), so we practically stop calculating it.
2015-04-10reimplement the last uses of sprintfDavid Kalnischkies
Working with strings c-style is complicated and error-prune, so by converting to c++ style we gain some simplicity and avoid buffer overflows by later extensions. Git-Dch: Ignore
2015-03-16stop displaying time of build in online helpJérémy Bobbio
As part of the “reproducible builds” effort [1], we have noticed that apt could not be built reproducibly. One issue is that it uses the __DATE__ and __TIME__ macros of the C preprocessor to display the time of build in the online help. We believe this information not to be really useful to users as they can always look at the package data and metadata to figure it out. The attached patch simply removes this information. All non-documentation packages can then be built reproducibly with our current experimental framework. [David: changed the string slightly to be untranslateable as well] Closes: 774342
2015-03-16merge debian/sid into debian/experimentalDavid Kalnischkies
2015-01-10128 KiB DSC files ought to be enough for everyoneDavid Kalnischkies
Your mileage may vary, but don't worry: There is more than one way to do it, but our one size fits all is not a bigger hammer, but an entire roundhouse kick! So brace yourself for the tl;dr: The limit is gone.* Beware: This fixes also the problem that a double newline is unconditionally added 'later' which is an overcommitment in case the dsc filesize is limit-2 <= x <= limit. * limited to numbers fitting into an unsigned long long. Closes: 774893
2014-11-09use the same code to detect quiet setting in all toolsDavid Kalnischkies
Git-Dch: Ignore
2014-11-09streamline display of --help in all toolsDavid Kalnischkies
By convention, if I run a tool with --help or --version I expect it to exit successfully with the usage, while if I do call it wrong (like without any parameters) I expect the usage message shown with a non-zero exit.
2014-10-24promote filesize to a hashstringDavid Kalnischkies
It is a very simple hashstring, which is why it isn't contributing to the usability of a list of them, but it is also trivial to check and calculate, so it doesn't hurt checking it either as it can combined even with the simplest other hashes greatly complicate attacks on them as you suddenly need a same-size hash collision, which is usually a lot harder to achieve.
2014-09-27fix: warning: extra ‘;’ [-Wpedantic]David Kalnischkies
Reported-By: gcc -Wpedantic Git-Dch: Ignore
2014-09-27fix: Member variable 'X' is not initialized in the constructor.David Kalnischkies
Reported-By: cppcheck Git-Dch: Ignore
2014-09-05Merge branch 'debian/sid' into debian/experimentalMichael Vogt
Conflicts: apt-pkg/acquire-item.cc configure.ac debian/changelog doc/apt-verbatim.ent doc/po/apt-doc.pot doc/po/de.po doc/po/es.po doc/po/fr.po doc/po/it.po doc/po/ja.po doc/po/pt.po po/ar.po po/ast.po po/bg.po po/bs.po po/ca.po po/cs.po po/cy.po po/da.po po/de.po po/dz.po po/el.po po/es.po po/eu.po po/fi.po po/fr.po po/gl.po po/hu.po po/it.po po/ja.po po/km.po po/ko.po po/ku.po po/lt.po po/mr.po po/nb.po po/ne.po po/nl.po po/nn.po po/pl.po po/pt.po po/pt_BR.po po/ro.po po/ru.po po/sk.po po/sl.po po/sv.po po/th.po po/tl.po po/tr.po po/uk.po po/vi.po po/zh_CN.po po/zh_TW.po test/integration/test-ubuntu-bug-346386-apt-get-update-paywall
2014-09-03Make Packages & Sources generation optional, during Generate callDimitri John Ledkov
refactor a bit, extract code out of Generate() into DoGenerate{PackagesAndSources,Contents}, add new APT::FTPArchive::ContentsOnly option to allow skipping the generation of Package/Source files (if they are generated e.g. by some db outside of apt-ftparchives control)
2014-06-18Merge branch 'debian/sid' into debian/experimentalMichael Vogt
Conflicts: debian/changelog
2014-06-04Add compat mode for old (32bit FileSize) CacheDB (LP: #1274466)Michael Vogt
2014-05-27use free() instead of delete() when realloc is usedMichael Vogt
ContentsExtract::~ContentsExtract() needs to use free() because Data got allocated via realloc() Reported-By: clang -fsanitize=address -fno-omit-frame-pointer
2014-05-09deal with hashes in ftparchive more dynamic as wellDavid Kalnischkies
Now that libapts acquire system happily passes around hashes and can be made to support new ones without an ABI break in the future, we can free ftparchive from all the deprecation warnings the last commit introduced for it. The goal here isn't to preserve ABI as we have none to keep here, but to help avoiding introduction problems of 'new' hashes later as bugs creep into the copy&paste parts, so short/less of them is good.
2014-05-08use a std::vector instead of hardcoded arrayDavid Kalnischkies
Arrays with predefined lengths are very fragile as they are likely forgotten in future changes and the size in this case is dynamic making it even more dangerous. Git-Dch: Ignore
2014-05-08Merge remote-tracking branch 'mvo/feature/apt-ftparchive-srccache2' into ↵Michael Vogt
debian/sid
2014-05-08fix regression from commit 215b0fafMichael Vogt
2014-05-07Merge remote-tracking branch 'mvo/feature/apt-ftparchive-srccache2' into ↵Michael Vogt
debian/sid
2014-04-25reduce delta from ubuntuMichael Vogt
2014-04-07ensure "--db" also works with the new srcpkgdbMichael Vogt
2014-04-04refactor _InitQuery()Michael Vogt
2014-04-04ensure clean worksMichael Vogt
2014-04-04refactor to make OpenFile/OpenDebFile more robustMichael Vogt
2014-04-04add test for binary cachedb and contents generationMichael Vogt
2014-04-04Implement CacheDB for source packages in apt-ftparchiveMichael Vogt
2014-03-13cleanup headers and especially #includes everywhereDavid Kalnischkies
Beside being a bit cleaner it hopefully also resolves oddball problems I have with high levels of parallel jobs. Git-Dch: Ignore Reported-By: iwyu (include-what-you-use)
2014-03-13warning: unused parameter ‘foo’ [-Wunused-parameter]David Kalnischkies
Reported-By: gcc -Wunused-parameter Git-Dch: Ignore