summaryrefslogtreecommitdiff
path: root/apt-pkg/contrib
AgeCommit message (Collapse)Author
2015-11-20do not segfault in cache generation on mmap failureDavid Kalnischkies
Out of memory and similar circumstanzas could cause MMap::Map to fail and especially the mmap/malloc calls in it. With some additional checking we can avoid segfaults and similar in such situations – at least in theory as if this is a real out of memory everything we do to handle the error could just as well run into a memory problem as well… But at least in theory (if MMap::Map is made to fail always) we can deal with it so good that a user actually never sees a failure (as the cache it tries to load with it fails and is discarded, so that DynamicMMap takes over and a new one is build) instead of segfaulting. Closes: 803417
2015-11-19support setting empty values (sanely) & removing support forDavid Kalnischkies
space-gapping: '-o option= value' That is a very old feature (straight from 1998), but it is super surprising if you try setting empty values and instead get error messages or a non-empty value as the next parameter is treated as the value – which could have been empty, so if for some reason you need a compatible way of setting an empty value try: '-o option="" ""'. I can only guess that the idea was to support '-o option value', but we survived 17 years without it, we will do fine in the future I guess. Similar is the case for '-t= testing' even through '-t testing' existed before and the code even tried to detect mistakes like '-t= -b' … all gone now. Technically that is as its removing a feature replacing it with another a major interface break. In practice I really hope for my and their sanity that nobody was using this; but if for some reaon you do: Remove the space and be done. I found the patch and the bugreport actually only after the fact, but its reassuring that others are puzzled by this as well and hence a thanks is in perfect order here as the patch is practical identical [expect that this one here adds tests and other bonus items]. Thanks: Daniel Hartwig for initial patch. Closes: 693092
2015-11-19do not use _apt for file/copy sources if it isn't world-accessibleDavid Kalnischkies
In 0940230d we started dropping privileges for file (and a bit later for copy, too) with the intend of uniforming this for all methods. The commit message says that the source will likely fail based on the compressors already – and there isn't much secret in the repository content. After all, after apt has run the update everyone can access the content via apt anyway… There are sources through which worked before which are mostly single-deb (and those with the uncompressed files available). The first one being especially surprising for users maybe, so instead of failing, we make it so that apt detects that it can't access a source as _apt and if so doesn't drop (for all sources!) privileges – but we limit this to file/copy, so the uncompress which might be needed will still fail – but that failed before this regression. We display a notice about this, mostly so that if it still fails (e.g. compressed) the user has some idea what is wrong. Closes: 805069
2015-11-05encode UTF-8 characters correctly in QuoteStringDavid Kalnischkies
Limit the field length to a char to avoid bogus FF for utf-8 characters with the default length. Closes: 799123
2015-11-04wrap every unlink call to check for != /dev/nullDavid Kalnischkies
Unlinking /dev/null is bad, we shouldn't do that. Also, we should print at least a warning if we tried to unlink a file but didn't manage to pull it of (ignoring the case were the file is /dev/null or doesn't exist in the first place). This got triggered by a relatively unlikely to cause problem in pkgAcquire::Worker::PrepareFiles which would while temporary uncompressed files (which are set to keep compressed) figure out that to files are the same and prepare for sharing by deleting them. Bad move. That also shows why not printing a warning is a bad idea as this hide the error for in non-root test runs. Git-Dch: Ignore
2015-11-04ensure FileFd doesn't try to open /dev/null as atomic and coDavid Kalnischkies
The wrapping will fail in the best case and actually end up deleting /dev/null in the worst case. Given that there is no point in trying to write atomically to /dev/null as you can't read from it again just ignore these flags if higher level code ends up trying to use them on /dev/null. Git-Dch: Ignore
2015-11-04move apts cmdline helper type into -privateDavid Kalnischkies
Its not as simple as I initially thought to abstract this enough to make it globally usable, so lets not pollute global namespace with this for now. Git-Dch: Ignore
2015-11-04new quiet level -qq for apt to hide progress outputDavid Kalnischkies
-q is for logging and -qqq (old -qq) basically kills every output expect errors, so there should be a way of declaring a middleground in which the output of e.g. 'update' isn't as verbose, but still shows some things. The test framework was actually making use of by accident as it ignored the quiet level in output setup for apt before. Eventually we should figure out some better quiet levels for all tools…
2015-11-04deduplicate main methodsDavid Kalnischkies
All mains pretty much do the same thing, so lets try a little harder to move the common parts into -private to have the real differences more visible. Git-Dch: Ignore
2015-11-04split up help messages for simpler reuseDavid Kalnischkies
That is one huge commit with busy work only: Help messages used to be one big translateable string, which is a pain for translators and hard to reuse for us. This change there 'explodes' this single string into new string for each documented string trying hard to split up the translated messages as well. This actually restores many translations as previously adding a single command made all of the bug message fuzzy. The splitup also highlighted that its easy to forget a line, duplicate one and similar stuff. Git-Dch: Ignore
2015-11-04add binary-specific options via Binary scopeDavid Kalnischkies
Especially with apt now, it can be useful to set an option only for apt and not for apt-get. Using a binary-specific subtree which is merged into the root seems like a simple enough trick to achieve this.
2015-10-30srvrec: Do not expose C++11 tuple use in headerJulian Andres Klode
This makes non-C++11 reverse deps wishing to use it FTBFS.
2015-10-30GetSrvRecords: Make thread-safeJulian Andres Klode
Gbp-Dch: ignore
2015-09-14srv test: do 100 pulls twice and compare listDavid Kalnischkies
The previous implementation was still a bit unstable in terms of failing at times. Lets try if we have more luck with this one. Git-Dch: Ignore
2015-09-14avoid using global PendingError to avoid failing too often too soonDavid Kalnischkies
Our error reporting is historically grown into some kind of mess. A while ago I implemented stacking for the global error which is used in this commit now to wrap calls to functions which do not report (all) errors via return, so that only failures in those calls cause a failure to propergate down the chain rather than failing if anything (potentially totally unrelated) has failed at some point in the past. This way we can avoid stopping the entire acquire process just because a single source produced an error for example. It also means that after the acquire process the cache is generated – even if the acquire process had failures – as we still have the old good data around we can and should generate a cache for (again). There are probably more instances of this hiding, but all these looked like the easiest to work with and fix with reasonable (aka net-positive) effects.
2015-09-14copy ReadWrite-error to the bottom to make clang happyDavid Kalnischkies
clang detects that fd isn't set in the ReadWrite case – just that this is supposed to be catched earlier in this method already, but it doesn't hurt to make it explicit here as well and clang is happy, too. Git-Dch: Ignore
2015-09-14implement CopyFile without using FileFd::Size()David Kalnischkies
Pipes and such have no good Size value, but we still want to copy from it maybe and we don't really need size as we can just as well read as long as we get data out of a file to copy it. Git-Dch: Ignore
2015-09-02avoid triggering the c++11 erase api change on travisDavid Kalnischkies
Git-Dch: Ignore
2015-09-01use clock() as source for SRV randomnessDavid Kalnischkies
Initializing a random number generator with the time since epoch could be good enough, but reaches its limits in test code as the 100 iterations might very well happen in the same second and hence the seed number is always the same… clock() has a way lower resolution so it changes more often and not unimportant: If many users start the update at the same time it isn't to unlikely the SRV record will be ordered in the same second choosing the same for them all, but it seems less likely that the exact same clock() time has passed for them. And if I have to touch this, lets change a few other things as well to make me and/or compilers a bit happier (clang complained about the usage of a GNU extension in the testcase for example).
2015-09-01use unusable-for-security hashes for integrity checksDavid Kalnischkies
We want to declare some hashes as not enough for security, so that a user will need --allow-unauthenticated or similar to get data secured only by those hashes, but we can still us these hashes for integrity checks if we got them.
2015-09-01Consider md5sum no longer a usable hashMichael Vogt
The md5sum hash is broken since some time and we should no longer consider it a usable hash. Also update the tests to reflect this.
2015-09-01improve CheckDropPrivsMustBeDisabled furtherDavid Kalnischkies
Various smaller improvements so that the check deals better with already downloaded files, relative paths and other things. Git-Dch: Ignore
2015-08-31ignore for _apt inaccessible TMPDIR in pkgAcqChangelogDavid Kalnischkies
Using libpam-tmpdir caused us to create our download tmp directory in root's private tmp before changing to _apt, which wouldn't have access to it. By extending our GetTempDir method with an optional wrapper changing the effective user, we can test if a given user can access the directory and ignore TMPDIR if not instead of ignoring TMPDIR completely. Closes: 797270
2015-08-27Always close compressed files in FileFdJulian Andres Klode
We dup() the file descriptor when opening compressed files, so we always need to close the dup()ed one. Furthermore, not unsetting the d-pointer causes issues when running OpenDescriptor() multiple times on the same file descriptor.
2015-08-20Add basic (non weight adjusted) shuffling for SrvRecords selectionMichael Vogt
Also add "Debug::Acquire::SrvRecs" debug option and the option "Acquire::EnableSrvRecods" to allow disabling this lookup.
2015-08-18apt-pkg/contrib/srvrec.cc: res_query() should not generate a _error->Warning()Michael Vogt
2015-08-18cleanupMichael Vogt
2015-08-18Merge remote-tracking branch 'upstream/debian/experimental' into ↵Michael Vogt
feature/srv-records
2015-08-18Merge branch 'debian/experimental' into feature/srv-recordsMichael Vogt
Conflicts: cmdline/apt-helper.cc cmdline/makefile
2015-08-14Also add 'in combination with the other options.' to another errorJulian Andres Klode
Gbp-Dch: ignore
2015-08-14Say "in combination with the other options" if an option is not understoodJulian Andres Klode
Closes: #762758
2015-08-13Use setresuid() and setresgid() where availableJulian Andres Klode
2015-08-13Deprecate SPtrArray<T> and convert everyone to unique_ptr<T[]>Julian Andres Klode
More standardization
2015-08-13Mark SPtr as deprecated, and convert users to std::unique_ptrJulian Andres Klode
Switch to std::unique_ptr, as this is safer than SPtr.
2015-08-11Annotate more methods with APT_OVERRIDEJulian Andres Klode
Gbp-Dch: ignore Reported-By: g++ -Wsuggest-override Thanks: g++ -Wsuggest-override
2015-08-11ExecFork: Use /proc/self/fd to determine which files to closeJulian Andres Klode
This significantly reduces the number of files that have to be closed and seems to be faster, despite the additional reads. On systems where /proc/self/fd is not available, we fallback to the old code that closes all file descriptors >= 3. Closes: #764204
2015-08-10change to libapt-pkg abi 5.0 with versioned symbolsDavid Kalnischkies
We changed an aweful lot of stuff, so 5.0 is properly better than 4.X as a semantic version and as we are at it lets add some trivial symbol versioning as well: We just mark all exported symbols with the same version for now. This isn't really the proper thing to do as if we add symbols in later versions (with the same abi) they will get the same symbols version, but our .symbols file will protect us from the problems arising from this as it will ensure that a package acutally depends on a version of the abi high enough to include the symbol.
2015-08-10elimate duplicated code in pkgIndexFile subclassesDavid Kalnischkies
Trade deduplication of code for a bunch of new virtuals, so it is actually visible how the different indexes behave cleaning up the interface at large in the process. Git-Dch: Ignore
2015-08-10remove the compatibility markers for 4.13 abiDavid Kalnischkies
We aren't and we will not be really compatible again with the previous stable abi, so lets drop these markers (which never made it into a released version) for good as they have outlived their intend already. Git-Dch: Ignore
2015-08-10add c++11 override marker to overridden methodsDavid Kalnischkies
C++11 adds the 'override' specifier to mark that a method is overriding a base class method and error out if not. We hide it in the APT_OVERRIDE macro to ensure that we keep compiling in pre-c++11 standards. Reported-By: clang-modernize -add-override -override-macros Git-Dch: Ignore
2015-08-10implement reverse_iterators for cachesetsDavid Kalnischkies
By further abstracting the iterator templates we can wrap the reverse iterators of the wrapped containers and share code in a way that iterator creating is now more template intensive, but shorter in code. Git-Dch: Ignore
2015-08-10implement Signed-By option for sources.listDavid Kalnischkies
Limits which key(s) can be used to sign a repository. Not immensely useful from a security perspective all by itself, but if the user has additional measures in place to confine a repository (like pinning) an attacker who gets the key for such a repository is limited to its potential and can't use the key to sign its attacks for an other (maybe less limited) repository… (yes, this is as weak as it sounds, but having the capability might come in handy for implementing other stuff later).
2015-08-10fix memory leaks reported by -fsanitizeDavid Kalnischkies
Various small leaks here and there. Nothing particularily big, but still good to fix. Found by the sanitizers while running our testcases. Reported-By: gcc -fsanitize Git-Dch: Ignore
2015-08-10some CXXFLAGS housekeepingDavid Kalnischkies
More warnings are always better. Git-Dch: Ignore
2015-08-10make all d-pointer * const pointersDavid Kalnischkies
Doing this disables the implicit copy assignment operator (among others) which would cause hovac if used on the classes as it would just copy the pointer, not the data the d-pointer points to. For most of the classes we don't need a copy assignment operator anyway and in many classes it was broken before as many contain a pointer of some sort. Only for our Cacheset Container interfaces we define an explicit copy assignment operator which could later be implemented to copy the data from one d-pointer to the other if we need it. Git-Dch: Ignore
2015-08-10apply various style suggestions by cppcheckDavid Kalnischkies
Some of them modify the ABI, but given that we prepare a big one already, these few hardly count for much. Git-Dch: Ignore
2015-06-11show URI.Path in all acquire item descriptionsDavid Kalnischkies
It is a rather strange sight that index items use SiteOnly which strips the Path, while e.g. deb files are downloaded with NoUserPassword which does not. Important to note here is that for the file transport Path is pretty important as there is no Host which would be displayed by Site, which always resulted in "interesting" unspecific errors for "file:". Adding a 'middle' ground between the two which does show the Path but potentially modifies it (it strips a pending / at the end if existing) solves this "file:" issue, syncs the output and in the end helps to identify which file is meant exactly in progress output and co as a single site can have multiple repositories in different paths.
2015-06-09configureable acquire targets to download additional filesDavid Kalnischkies
First pass at making the acquire system capable of downloading files based on configuration rather than hardcoded entries. It is now possible to instruct 'deb' and 'deb-src' sources.list lines to download more than just Packages/Translation-* and Sources files. Details on how to do that can be found in the included documentation file.
2015-06-09support hashes for compressed pdiff filesDavid Kalnischkies
At the moment we only have hashes for the uncompressed pdiff files, but via the new '$HASH-Download' field in the .diff/Index hashes can be provided for the .gz compressed pdiff file, which apt will pick up now and use to verify the download. Now, we "just" need a buy in from the creators of repositories…
2015-06-09rework hashsum verification in the acquire systemDavid Kalnischkies
Having every item having its own code to verify the file(s) it handles is an errorprune process and easy to break, especially if items move through various stages (download, uncompress, patching, …). With a giant rework we centralize (most of) the verification to have a better enforcement rate and (hopefully) less chance for bugs, but it breaks the ABI bigtime in exchange – and as we break it anyway, it is broken even harder. It shouldn't effect most frontends as they don't deal with the acquire system at all or implement their own items, but some do and will need to be patched (might be an opportunity to use apt on-board material). The theory is simple: Items implement methods to decide if hashes need to be checked (in this stage) and to return the expected hashes for this item (in this stage). The verification itself is done in worker message passing which has the benefit that a hashsum error is now a proper error for the acquire system rather than a Done() which is later revised to a Failed().