summaryrefslogtreecommitdiff
path: root/methods/rred.cc
AgeCommit message (Collapse)Author
2014-01-21methods/rred: minor robustness improvementsAnthony Towns
Use retry_fwrite to better handle partial fwrite successes, and to keep the Hashes in sync with what's actually written.
2014-01-15integrate Anthonys rred with POC for client-side mergeDavid Kalnischkies
Providing the benefits of both without the downsides :) (ABI breaks or external dependencies) For this Anthonys rred is equipped with: - magic-filename-pickup of patches rather than explicit messages - use of FileFd instead of FILE* to get on-the-fly uncompress of the gzip compressed pdiff patches The acquire code in turn stops checking for apt-file's helper as our own rred is now clever enough for our needs.
2014-01-15reimplement rred to allow applying all the diffs in a single passAnthony Towns
Based on the idea presented in: https://lists.debian.org/deity/2009/08/msg00169.html and https://lists.debian.org/debian-devel/2014/01/msg00081.html It reads all patches one by one and merges them in-memory before applying the merged changes to the index. Beware: This commit by David Kalnischkies rips out the rred binary rewrite unchanged (expect minor format issue corrections) from the proposed changes, so this commit alone BREAKS pdiff completely. The integration into the acquire system as it was prepared in the previous POC will be done in the next commit to have proper 'blame'.
2013-12-13implement POC client-side merging of pdiffs via apt-fileDavid Kalnischkies
The idea of pdiffs is to avoid downloading the hole file by patching the existing index. This works very well, but becomes slow if a lot of patches needs to be applied to reconstruct an up-to-date index and in recent years more and more dinstall (or similar) runs are executed creating more and more pdiffs in the same amount of time, so pdiffs became less useful. The solution is simple: Reduce the amount of patches (which are very small) which need to be applied on top of the index we have available (which is usually pretty big). This can be done in two ways: Either merge the patches on the server-side so that the client has to download only one patch or the patches are all downloaded and merged on the client-side. The first needs a client who is doing one step at a time who can also skip patches if it needs (APT supports this for a long time now). The later is implemented by this commit, but depends on the server NOT merging the patches and the patches being in a strict order in which no patch is skipped. This is traditionally the case for dak, but other repository creators support merging – e.g. reprepro (which helpfully adds a flag indicating that the patches are merged). To support both or even mixes a client needs more information which isn't available for now. This POC uses the external diffindex-rred included in apt-file to do the heavy lifting of merging & applying all patches in one pass, hence to test this feature apt-file needs to be installed.
2012-05-10we don't need zlib (anymore) in rred so don't include itDavid Kalnischkies
2012-03-22make these retry_write methods static so that they don't end up as symbolsDavid Kalnischkies
2012-03-20* methods/rred.cc:David Kalnischkies
- check return of writev() as gcc recommends * methods/mirror.cc: - check return of chdir() as gcc recommends * apt-pkg/deb/dpkgpm.cc: - check return of write() a gcc recommends * apt-inst/deb/debfile.cc: - check return of chdir() as gcc recommends * apt-inst/deb/dpkgdb.cc: - check return of chdir() as gcc recommends
2012-01-20fix a few esoteric cppcheck errors/warnings/infosDavid Kalnischkies
2012-01-10as Size() can be quiet expensive for compressed files lets store the resultDavid Kalnischkies
2011-12-18implement the fallback method of rred by using the FileFd and the includedDavid Kalnischkies
ReadLine instead of accessing the files directly with fgets()
2011-12-11 - add a ReadLine methodDavid Kalnischkies
- drop the explicit export of gz-compression handling
2011-12-10enable FileFd to guess the compressor based on the filename if requested orDavid Kalnischkies
to search for compressed silbings of the given filename and use this guessing instead of hardcoding Gzip compression
2011-09-19use forward declaration in headers if possible instead of includesDavid Kalnischkies
2011-09-19do not pollute namespace in the headers with using (Closes: #500198)David Kalnischkies
2011-09-13merge with debian/experimentalDavid Kalnischkies
2011-09-13Support large files in the complete toolset. Indexes of thisDavid Kalnischkies
size are pretty unlikely for now, but we need it for deb packages which could become bigger than 4GB now (LP: #815895)
2011-09-13reorder includes: add <config.h> if needed and include it at firstDavid Kalnischkies
2011-08-11follow the recommendation of cppcheck to make some method methods (scnr)David Kalnischkies
const and initial mostly Debug member values in the constructors
2011-02-14* apt-pkg/contrib/fileutl.cc:David Kalnischkies
- really detect bigendian machines by including config.h, so we can really (Closes: #612986) * apt-pkg/contrib/mmap.cc: - Base has as 'valid' failure states 0 and -1 so add a simple validData method to check for failure states
2011-02-13update size of dynamic MMap as we write in from the outsideDavid Kalnischkies
2011-02-12* methods/rred.cc:David Kalnischkies
- read patch into MMap only if we work on uncompressed patches
2011-01-15* methods/rred.cc:David Kalnischkies
- operate optional on gzip compressed pdiffs * apt-pkg/acquire-item.cc: - don't uncompress downloaded pdiff files before feeding it to rred
2010-08-10apt-pkg, methods: Convert users of WriteEmpty to WriteAtomic.Julian Andres Klode
2010-05-04* methods/rred.cc:David Kalnischkies
- use the patchfile modification time instead of the one from the "old" file - thanks to Philipp Weis for noticing! (Closes: #571541)
2009-12-11Backport rred patches from my own sid branch to the 0.7.25 branchDavid Kalnischkies
* rewrite and refactor rred method to be able to handle even big (>30 MB) patches (Closes: #554349) and hardening the method itself by using more constants and a return value which can't be misinterpreted as linenumber * Finally adope the patch from Morten Hustveit <morten@debian.org> to be able to optional use mmaps and iovec to increase patch speed - but as this increase memory usage we can always fall back to the "old" method which doesn't depend on mmaps.
2009-09-08add text at the top without failing in rred (Closes: #545694)David Kalnischkies
Patch from Bernhard R. Link, thanks!
2005-08-29* debug support (Debug::pkgAcquire::RRed) for rred method addedMichael Vogt
2005-08-23* method/rred.cc cleanup (thanks to robertle)Michael Vogt
2005-08-22* bugfix/updates in the rred.cc code (big thanks for helping Robert!)Michael Vogt
2005-08-22* rred.cc: use the d'tor of FileFd to close the file-fd; acquuire-item.cc: ↵Michael Vogt
make the QueueNextDiff() code more robust
2005-08-19* use the new cool rred method for the patchtingMichael Vogt
2005-07-28* added methods/rred.cc (thanks to robertle)Michael Vogt