* merge of the debian-expermental-ma branch

  
* debian/control:
  - add dependency on zlib-dev for libapt-pkg-dev
* apt-pkg/cacheset.cc:
  - [ABI BREAK] add an ErrorType option to CacheSetHelper
* cmdline/apt-cache.cc:
  - use Notice instead of Error in the CacheSetHelper messages
    for compat reasons. Otherwise tools like sbuild blow up
  - return success in show if a virtual package was given
* debian/control:
  - remove libcurl3-gnutls-dev alternative as the package is gone
  - increase needed version of libcurl4-gnutls-dev to >= 7.19.0
    as we use CURLOPT_{ISSUERCERT,CRLFILE} (Closes: #589642)
* apt-pkg/deb/dpkgpm.cc:
  - Write architecture information to history file.
  - Add to history whether a change was automatic or not.
* apt-pkg/contrib/fileutl.cc:
  - Add FileFd::OpenDescriptor() (needed for python-apt's #383617).
* cmdline/apt-get.cc:
  - Support large filesystems by using statvfs64() instead of statvfs()
    and statfs64() instead of statfs() (Closes: #590513).
* apt-pkg/cdrom.cc:
  - Use link() instead of rename() for creating the CD database backup;
    otherwise there would be a short time without any database.
* apt-pkg/depcache.cc:
  - handle "circular" conflicts for "all" packages correctly
* cmdline/apt-cache.cc:
  - be able to omit dependency types in (r)depends (Closes: #319006)
  - show in (r)depends the canidate per default instead of newest
  - share the (r)depends code instead of codecopy
* apt-pkg/cacheset.cc:
  - move them back to the library as they look stable now
  - add a 'newest' pseudo target release as in pkg/newest
* apt-pkg/pkgcache.cc:
  - prefer non-virtual packages in FindPreferredPkg (Closes: #590041)
* test/integration/*:
  - add with bug#590041 testcase a small test "framework"
* apt-pkg/orderlist.cc:
  - try to install another or-group member in DepRemove before
    breaking the or group (Closes: #590438)
  - configure also the replacement before remove by adding Immediate flag

* apt-pkg/contrib/error.{cc,h}
  - docstring cleanup
  - add inline DumpError() to avoid subtle API break
* apt-pkg/contrib/error.{cc,h}:
  - remove constness of va_list parameter to fix build on amd64 and co
    Thanks Eric Valette! (Closes: #588610)
* apt-pkg/deb/debmetaindex.cc:
  - do not query each architecture for flat file archives
  - fix typo preventing display of architecture in Info()
* methods/bzip2.cc:
  - add a copycat of the old gzip.cc as we need it for bzip2 and lzma
* debian/rules:
  - Make DEB_BUILD_OPTIONS=noopt actually work by passing the right
    CXXFLAGS.
* apt-pkg/contrib/fileutl.{h,cc}:
  - Add support for reading of gzipped files with the new "ReadOnlyGzip"
    OpenMode. (Closes: #188407)
  - Link against zlib (in apt-pkg/makefile) and add zlib build dependency.
  - [ABI BREAK] This adds a new private member to FileFd, but its
    initialization is in the public header file.
* configure.in:
  - Check for zlib library and headers.
* apt-pkg/acquire-item.cc, apt-pkg/deb/debindexfile.cc,
  apt-pkg/deb/debrecords.cc, apt-pkg/deb/debsrcrecords.h,
  cmdline/apt-cache.cc:
  - Open Packages, Sources, and Translations indexes in "ReadOnlyGzip" mode.
* apt-pkg/deb/debindexfile.cc:
  - If we do not find uncompressed package/source/translation indexes, look
    for gzip compressed ones.
* apt-pkg/acquire-item.cc:
  - If the Acquire::GzipIndexes option is true and we download a gzipped
    index file, keep it as it is (and rename to .gz) instead of
    uncompressing it.
* doc/apt.conf.5.xml:
  - Document the new Acquire::GzipIndexes option.
* doc/po/apt-doc.pot, doc/po/de.po:
  - German translation of new Acquire::GzipIndexes option.
* Add test/test-indexes.sh:
  - Test behaviour of index retrieval and usage, in particular with
    uncompressed and gzip compressed indexes.
* methods/gzip.cc: With FileFd now being able to read gzipped files, there
  is no need for the gzip method any more to spawn an external gzip process.
  Rewrite it to use FileFd directly, which makes the code a lot simpler, and
  also using less memory and overhead.
* doc/apt.conf.5.xml:
  - add and document APT::Cache-{Start,Grow,Limit} options for mmap control
* apt-pkg/contrib/fileutl.cc:
  - do not fail von double close()
* cmdline/cacheset.cc:
  - doesn't include it in the library for now as it is too volatile
  - get the candidate either from an already built depcache
    or use the policy which is a bit faster than depcache generation
  - get packages by task^ with FromTask()
  - only print errors if all tries to get a package by string failed
  - factor out code to get a single package FromName()
  - check in Grouped* first without modifier interpretation
* cmdline/apt-get.cc:
  - use the cachsets in the install commands
  - make the specify order of packages irrelevant (Closes: #196021)
* apt-pkg/orderlist.cc:
  - untouched packages are never missing
* apt-pkg/packagemanager.cc:
  - packages that are not touched doesn't need to be unpacked
* debian/control:
  - remove intltool's dependency as it is an ubuntu artefact
* apt-pkg/depcache.cc:
  - SetCandidateVer for all pseudo packages
  - SetReInstall for the "all" package of a pseudo package
  - use the new MatchAgainstConfig for the DefaultRootSetFunc
  - always mark the all package if a pseudo package is marked for install
* apt-pkg/contrib/error.{cc,h}:
  - complete rewrite but use the same API
  - add NOTICE and DEBUG as new types of a message
  - add a simple stack handling to be able to delay error handling
* apt-pkg/aptconfiguration.cc:
  - show a deprecation notice for APT::Acquire::Translation
* apt-pkg/contrib/configuration.{cc,h}:
  - add a wrapper to match strings against configurable regex patterns
* apt-pkg/contrib/fileutl.cc:
  - show notice about ignored file instead of being always silent
  - add a Dir::Ignore-Files-Silently list option to control the notice
* apt-pkg/policy.h:
  - add another round of const& madness as the previous round accidentally
    NOT overrides the virtual GetCandidateVer() method (Closes: #587725)
* apt-pkg/pkgcachegen.{cc,h}:
  - make the used MMap moveable (and therefore dynamic resizeable) by
    applying (some) mad pointer magic (Closes: #195018)
* apt-pkg/deb/dpkgpm.cc:
  - make the apt/term.log output unbuffered (thanks to Matt Zimmerman)
* methods/ftp.h:
  - Handle different logins on the same server (Closes: #586904).
* apt-pkg/deb/deblistparser.cc:
  - Handle architecture wildcards (Closes: #547724).
* apt-pkg/versionmatch.cc:
  - Support matching pins by regular expressions or glob() like patterns,
    regular expressions have to be put between to slashes; for example,
    /.*/.
* apt-pkg/contrib/fileutl.cc:
  - Make FileFd replace files atomically in WriteTemp mode (for cache, etc).
* debian/control:
  - Set Standards-Version to 3.9.0
* apt-pkg/cachefile.h:
  - make pkgPolicy public again, libapt-pkg-perl (and probably
    others) get unhappy without that
* merge the remaining Ubuntu change:
  - on gpg verification failure warn and restore the last known
    good state
  - on failure display the IP of the server (useful for servers
    that use round robin DNS)
  - support Original-Maintainer in RewritePackageOrder
  - enable cdrom autodetection via libudev by default
  - show message about Vcs in use when apt-get source is run for
    packages maintained in a Vcs
  - better support transitional packages with mark auto-installed. 
    when the transitional package is in "oldlibs" the new package
    is not marked auto installed (same is true for section
    metapackages)
  - provide new "deb mirror://archive.foo/mirrors.list sid main"
    method expects a list of mirrors (generated on the server e.g.
    via geoip) and will use that, including cycle on failure
  - write apport crash file on package failure (disabled by default
    on debian until apport is available)
  - support mirror failure reporting (disabled by default on debian)

* apt-pkg/deb/dpkgpm.cc:
  - write Disappeared also to the history.log
  - forward manual-installed bit on package disappearance
* apt-pkg/deb/debsystem.cc:
  - add better config item for extended_states file
* apt-pkg/pkgcache.h:
  - switch {,Install-}Size to unsigned long long
* apt-pkg/depcache.cc:
  - do the autoremove mark process also for required packages to handle
    these illegally depending on lower priority packages (Closes: #583517)
  - try harder to find the other pseudo versions for autoremove multiarch
  - correct "Dangerous iterator usage" pointed out by cppcheck
  - deal with long long, not with int to remove 2GB Limit (LP: #250909)
  - deprecate AddSize with Multiplier as it is unused and switch to
    boolean instead to handle the sizes more gracefully.
  - switch i{Download,Usr}Size from double to (un)signed long long
* apt-pkg/aptconfiguration.cc:
  - remove duplicate architectures in getArchitectures()
* apt-pkg/indexrecords.{cc,h}:
  - backport forgotten Valid-Until patch from the obsolete experimental
    branch to prevent replay attacks better, thanks to Thomas Viehmann
    for the initial patch! (Closes: #499897)
  - add a constant Exists check for MetaKeys
* apt-pkg/acquire-item.cc:
  - do not try PDiff if it is not listed in the Meta file
  - sent Last-Modified header also for Translation files
* apt-pkg/cacheiterator.h:
  - let pkgCache::Iterator inherent std::iterator
* ftparchive/writer.h:
  - add a virtual destructor to FTWScanner class (for cppcheck)
* apt-pkg/cacheset.{cc,h}:
  - add simple wrapper around std::set for cache structures
  - move regex magic from apt-get to new FromRegEx method
  - move cmdline parsing from apt-cache to new FromCommandLine method
  - support special release-modifier 'installed' and 'candidate'
* apt-pkg/contrib/cmdline.cc:
  - fix segfault in SaveInConfig caused by writing over char[] sizes
* apt-pkg/pkgcache.cc:
  - get the best matching arch package from a group with FindPreferredPkg
* cmdline/apt-cache.cc:
  - make the search multiarch compatible by using GrpIterator instead
  - use pkgCacheFile and the new CacheSets all over the place
  - add --target-release option (Closes: #115520)
  - accept pkg/release and pkg=version in show and co. (Closes: #236270)
  - accept package versions in the unmet command
* cmdline/apt-get.cc:
  - use unsigned long long instead of double to store values it gets
* apt-pkg/cachefile.{cc,h}:
  - split Open() into submethods to be able to build only parts
  - make the OpProgress optional in the Cache buildprocess
  - store also the SourceList we use internally for export
* doc/apt.conf.5.xml:
  - document the new Valid-Until related options
* apt-pkg/contrib/strutl.cc:
  - split StrToTime() into HTTP1.1 and FTP date parser methods and
    use strptime() instead of some self-made scanf mangling
  - use the portable timegm shown in his manpage instead of a strange
    looking code copycat from wget
* ftparchive/writer.cc:
  - add ValidTime option to generate a Valid-Until header in Release file
* apt-pkg/policy.cc:
  - get the candidate right for a not-installed pseudo package if
    his non-pseudo friend is installed
* apt-pkg/indexcopy.cc:
  - move the gpg codecopy to a new method and use it also in methods/gpgv.cc
* cmdline/apt-get.cc:
  - rerun dpkg-source in source if --fix-broken is given (Closes: #576752)
  - don't suggest held packages as they are installed (Closes: #578135)
  - handle multiple --{tar,diff,dsc}-only options correctly
  - show at the end of the install process a list of disappeared packages
* cmdline/apt-cache.cc:
  - use GroupCount for package names in stats and add a package struct line
* methods/rred.cc:
  - use the patchfile modification time instead of the one from the
    "old" file - thanks to Philipp Weis for noticing! (Closes: #571541)
* debian/rules:
  - remove targets referring to CVS or arch as they are useless
  - use $(CURDIR) instead of $(pwd)
  - use dpkg-buildflags if available for CXXFLAGS
* README.arch:
  - remove the file completely as it has no use nowadays
* apt-pkg/depcache.cc:
  - be doublesure that the killer query is empty before starting reinstall
* methods/gpgv.cc:
  - remove the keyrings count limit by using vector magic
* contrib/mmap.cc:
  - clarify "MMap reached size limit" error message, thanks Ivan Masár!
* doc/apt.ent
  - add entities for the current oldstable/stable/testing codenames
* doc/sources.list.5.xml:
  - use stable-codename instead of stable in the examples (Closes: #531492)
* doc/apt_preferences.5.xml:
  - adapt some examples here to use current codenames as well
  - add "NotAutomatic: yes" handling, thanks Osamu Aoki (Closes: #490347)
* debian/libapt-pkg-doc.doc-base.cache:
  - remove yet another reference to the removed cache.sgml
* doc/apt-get.8.xml:
  - do not say explicit target_release_{name,version,codename}, it should
    be clear by itself and 'man' can break lines again (Closes: #566166)
  - remove the gnome-apt reference as it is removed from unstable
* apt-pkg/deb/dpkgpm.cc:
  - add 'disappear' to the known processing states, thanks Jonathan Nieder
* apt-pkg/packagemanager.h:
  - export info about disappeared packages with GetDisappearedPackages()
* methods/http.{cc,h}:
  - code cleanup, use enums instead of magic ints

* debian/rules:
  - spell out some less known options to reduce manpage consultation-rate
  - Use POSIX command substitution: $(<command sequence>)
  - Remove EOL whitespace (Closes: #577804)
* apt-pkg/acquire-item.cc:
  - Fix pkgAcqFile::Custom600Headers() to always return something.

* apt-pkg/depcache.cc:
  - rewrite the pseudo package reinstaller to be more intelligent
    in his package choices
* apt-pkg/packagemanager.cc:
  - don't try to "unpack" pseudo packages twice
* apt-pkg/contrib/fileutl.cc:
  - add a parent-guarded "mkdir -p" as CreateDirectory()
* apt-pkg/acquire.{cc,h}:
  - add a delayed constructor with Setup() for success reporting
  - check for and create directories in Setup if needed instead of
    error out unfriendly in the Constructor (Closes: #523920, #525783)
  - optional handle a lock file in Setup()
* apt-pkg/acquire-item.cc:
  - Acquire::ForceHash to force method for expected hash
* cmdline/apt-get.cc:
  - remove the lock file handling and let Acquire take care of it instead
  - display MD5Sum in --print-uris if not forced to use another method
    instead of displaying the strongest available (Closes: #576420)
  - regex for package names executed on Grp- not PkgIterator
  - show non-candidates as fallback for virtual packages (Closes: #578385)
  - set also "all" to this version for pseudo packages in TryToChangeVer
* apt-pkg/deb/dpkgpm.cc:
  - remove Chroot-Directory from files passed to install commands.
    Thanks to Kel Modderman for report & patch! (Closes: #577226)
* ftparchive/writer.cc:
  - remove 999 chars Files and Checksums rewrite limit (Closes: #577759)
* cmdline/apt-cache.cc:
  - align Installed and Candidate Version in policy so they can be compared
    easier, thanks Ralf Gesellensetter for the pointer! (Closes: #578657)
* doc/apt.ent:
  - Add a note about APT_CONFIG in the -c description (Closes: #578267)
* doc/po/de.po:
  - correct typos in german apt_preferences manpage, thanks Chris Leick!
* apt-pkg/sourcelist.cc:
  - be less strict and accept [option=value] as well
* apt-pkg/contrib/configuration.cc:
  - error out if #clear directive has no argument
* doc/files.sgml:
  - sync documentation with status quo, regarding files/directories in
    use, extended_states and uri schemes.
* doc/cache.sgml:
  - drop the file in favor of inplace documentation with doxygen
* apt-pkg/pkgcache.h:
  - enhance the Groups ABI by providing a ID as the other structs does
  - check also the size of the Group struct then checking for the others
* cmdline/apt-get.cc:
  - replace backticks with single quotes around fix-broken command
    in the broken packages message. (Closes: #577168)
* dselect/install:
  - modernize if-statements not to use 'x' (Closes: #577117)
  - replace backticks with POSIX $() (Closes: #577116)
* cmdline/apt-get.cc:
  - fix crash when pkg.VersionList() is empty
  - install html doxygen in libapt-pkg-doc 
* debian/control:
  - build-depend on doxygen
* apt-pkg/contrib/weakptr.h:
  - add a class WeakPointable which allows one to register weak pointers to
    an object which will be set to NULL when the object is deallocated.
* [ABI break] apt-pkg/acquire{-worker,-item,}.h:
  - subclass pkgAcquire::{Worker,Item,ItemDesc} from WeakPointable.
* apt-pkg/pkgcache.cc:
  - Merge fix from David to correct handling in single-arch environments.
* cmdline/apt-cache.cc:
  - Add a showauto command to apt-cache.
* cmdline/apt-get.cc:
  - Add apt-get markauto and unmarkauto commands.
* [BREAK] merge MultiArch-ABI. We don't support MultiArch,
  but we support the usage of the new ABI so libapt users
  can start to prepare for MultiArch (Closes: #536029)
* Ignore :qualifiers after package name in build dependencies
  in the library by default, but try to honour them in apt-get
  as we have some sort of MultiArch support ready (Closes: #558103)
* add translation of the manpages to PT (portuguese)
  Thanks to Américo Monteiro!
* Switch to dpkg-source 3.0 (native) format
* apt-pkg/depcache.cc:
  - remove Auto-Installed information from extended_states
    together with the package itself (Closes: #572364)
* cmdline/apt-mark:
  - don't crash if no arguments are given (Closes: #570962)
* debian/control:
  - remove some years old and obsolete Replaces
  - add automake/conf build-depends/conflicts as recommend by
    the autotools-dev README (Closes: #572615)
* apt-pkg/contrib/mmap.{h,cc}:
  - add char[] fallback for filesystems without shared writable
    mmap() like JFFS2. Thanks to Marius Vollmer for writing
    and to Loïc Minier for pointing to the patch! (Closes: #314334)
* doc/apt_preferences.5.xml:
  - fix two typos and be more verbose in the novice warning.
    Thanks to Osamu Aoki for pointing it out! (Closes: #567669)
  - fix a=sid vs. n=sid typo, thanks Ansgar Burchardt!
  - origin can be used to match a hostname (Closes: #352667)
  - remove wrong pin-priority is optional remark (Closes: #574944)
* apt-pkg/deb/dpkgpm.cc:
  - fix error message construction in OpenLog()
  - if available store the Commandline in the history
* cmdline/apt-get.cc:
  - add a --only-upgrade flag to install command (Closes: #572259)
  - fix memory leaks in error conditions in DoSource()
  - try version match in FindSrc first exact than fuzzy (LP: #551178)
* apt-pkg/contrib/cmndline.cc:
  - save Commandline in Commandline::AsString for logging
* apt-pkg/deb/debversion.cc:
  - consider absent of debian revision equivalent to 0 (Closes: #573592)
* doc/makefile, doc/*:
  - generate subdirectories for building the manpages in on the fly
    depending on the po files we have.
* apt-pkg/pkgcachegen.cc:
  - merge versions correctly even if multiple different versions
    with the same version number are available.
    Thanks to Magnus Holmgren for the patch! (Closes: #351056)
* ftparchive/writer.cc:
  - write LongDescriptions if they shouldn't be included in Packages
    file into i18n/Translation-en by default.
* doc/po/de.po:
  - correct a few typos in the german manpage translation.
    Thanks to Chris Leick and Georg Koppen! (Closes: #574962)
* apt-pkg/contrib/strutl.cc:
  - convert all toupper calls to tolower_ascii for a little speedup
* apt-pkg/contrib/strutl.cc:
  - always escape '%' (LP: #130289) (Closes: #500560)
  - unescape '%' sequence only if followed by 2 hex digit
  - username/password are urlencoded in proxy string (RFC 3986)

1 files changed, 460 insertions, 139 deletions

diff --git a/apt-pkg/pkgcachegen.cc b/apt-pkg/pkgcachegen.cc
index 3eeb18cae..c9a9a753c 100644
--- a/apt-pkg/pkgcachegen.cc
+++ b/apt-pkg/pkgcachegen.cc
@@ -18,6 +18,7 @@
 #include <apt-pkg/progress.h>
 #include <apt-pkg/sourcelist.h>
 #include <apt-pkg/configuration.h>
+#include <apt-pkg/aptconfiguration.h>
 #include <apt-pkg/strutl.h>
 #include <apt-pkg/sptr.h>
 #include <apt-pkg/pkgsystem.h>
@@ -35,10 +36,11 @@
 #include <stdio.h>
 									/*}}}*/
 typedef vector<pkgIndexFile *>::iterator FileIterator;
+template <typename Iter> std::vector<Iter*> pkgCacheGenerator::Dynamic<Iter>::toReMap;
 
 // CacheGenerator::pkgCacheGenerator - Constructor			/*{{{*/
 // ---------------------------------------------------------------------
-/* We set the diry flag and make sure that is written to the disk */
+/* We set the dirty flag and make sure that is written to the disk */
 pkgCacheGenerator::pkgCacheGenerator(DynamicMMap *pMap,OpProgress *Prog) :
 		    Map(*pMap), Cache(pMap,false), Progress(Prog),
 		    FoundFileDeps(0)
@@ -60,8 +62,12 @@ pkgCacheGenerator::pkgCacheGenerator(DynamicMMap *pMap,OpProgress *Prog) :
 
       // Starting header
       *Cache.HeaderP = pkgCache::Header();
-      Cache.HeaderP->VerSysName = Map.WriteString(_system->VS->Label);
-      Cache.HeaderP->Architecture = Map.WriteString(_config->Find("APT::Architecture"));
+      map_ptrloc const idxVerSysName = WriteStringInMap(_system->VS->Label);
+      Cache.HeaderP->VerSysName = idxVerSysName;
+      map_ptrloc const idxArchitecture = WriteStringInMap(_config->Find("APT::Architecture"));
+      Cache.HeaderP->Architecture = idxArchitecture;
+      if (unlikely(idxVerSysName == 0 || idxArchitecture == 0))
+	 return;
       Cache.ReMap();
    }
    else
@@ -95,6 +101,67 @@ pkgCacheGenerator::~pkgCacheGenerator()
    Map.Sync(0,sizeof(pkgCache::Header));
 }
 									/*}}}*/
+void pkgCacheGenerator::ReMap(void const * const oldMap, void const * const newMap) {/*{{{*/
+   if (oldMap == newMap)
+      return;
+
+   Cache.ReMap(false);
+
+   CurrentFile += (pkgCache::PackageFile*) newMap - (pkgCache::PackageFile*) oldMap;
+
+   for (size_t i = 0; i < _count(UniqHash); ++i)
+      if (UniqHash[i] != 0)
+	 UniqHash[i] += (pkgCache::StringItem*) newMap - (pkgCache::StringItem*) oldMap;
+
+   for (std::vector<pkgCache::GrpIterator*>::const_iterator i = Dynamic<pkgCache::GrpIterator>::toReMap.begin();
+	i != Dynamic<pkgCache::GrpIterator>::toReMap.end(); ++i)
+      (*i)->ReMap(oldMap, newMap);
+   for (std::vector<pkgCache::PkgIterator*>::const_iterator i = Dynamic<pkgCache::PkgIterator>::toReMap.begin();
+	i != Dynamic<pkgCache::PkgIterator>::toReMap.end(); ++i)
+      (*i)->ReMap(oldMap, newMap);
+   for (std::vector<pkgCache::VerIterator*>::const_iterator i = Dynamic<pkgCache::VerIterator>::toReMap.begin();
+	i != Dynamic<pkgCache::VerIterator>::toReMap.end(); ++i)
+      (*i)->ReMap(oldMap, newMap);
+   for (std::vector<pkgCache::DepIterator*>::const_iterator i = Dynamic<pkgCache::DepIterator>::toReMap.begin();
+	i != Dynamic<pkgCache::DepIterator>::toReMap.end(); ++i)
+      (*i)->ReMap(oldMap, newMap);
+   for (std::vector<pkgCache::DescIterator*>::const_iterator i = Dynamic<pkgCache::DescIterator>::toReMap.begin();
+	i != Dynamic<pkgCache::DescIterator>::toReMap.end(); ++i)
+      (*i)->ReMap(oldMap, newMap);
+   for (std::vector<pkgCache::PrvIterator*>::const_iterator i = Dynamic<pkgCache::PrvIterator>::toReMap.begin();
+	i != Dynamic<pkgCache::PrvIterator>::toReMap.end(); ++i)
+      (*i)->ReMap(oldMap, newMap);
+   for (std::vector<pkgCache::PkgFileIterator*>::const_iterator i = Dynamic<pkgCache::PkgFileIterator>::toReMap.begin();
+	i != Dynamic<pkgCache::PkgFileIterator>::toReMap.end(); ++i)
+      (*i)->ReMap(oldMap, newMap);
+}									/*}}}*/
+// CacheGenerator::WriteStringInMap					/*{{{*/
+map_ptrloc pkgCacheGenerator::WriteStringInMap(const char *String,
+					const unsigned long &Len) {
+   void const * const oldMap = Map.Data();
+   map_ptrloc const index = Map.WriteString(String, Len);
+   if (index != 0)
+      ReMap(oldMap, Map.Data());
+   return index;
+}
+									/*}}}*/
+// CacheGenerator::WriteStringInMap					/*{{{*/
+map_ptrloc pkgCacheGenerator::WriteStringInMap(const char *String) {
+   void const * const oldMap = Map.Data();
+   map_ptrloc const index = Map.WriteString(String);
+   if (index != 0)
+      ReMap(oldMap, Map.Data());
+   return index;
+}
+									/*}}}*/
+map_ptrloc pkgCacheGenerator::AllocateInMap(const unsigned long &size) {/*{{{*/
+   void const * const oldMap = Map.Data();
+   map_ptrloc const index = Map.Allocate(size);
+   if (index != 0)
+      ReMap(oldMap, Map.Data());
+   return index;
+}
+									/*}}}*/
 // CacheGenerator::MergeList - Merge the package list			/*{{{*/
 // ---------------------------------------------------------------------
 /* This provides the generation of the entries in the cache. Each loop
@@ -107,13 +174,27 @@ bool pkgCacheGenerator::MergeList(ListParser &List,
    unsigned int Counter = 0;
    while (List.Step() == true)
    {
-      // Get a pointer to the package structure
-      string PackageName = List.Package();
+      string const PackageName = List.Package();
       if (PackageName.empty() == true)
 	 return false;
-      
+
+      /* As we handle Arch all packages as architecture bounded
+         we add all information to every (simulated) arch package */
+      std::vector<string> genArch;
+      if (List.ArchitectureAll() == true) {
+	 genArch = APT::Configuration::getArchitectures();
+	 if (genArch.size() != 1)
+	    genArch.push_back("all");
+      } else
+	 genArch.push_back(List.Architecture());
+
+      for (std::vector<string>::const_iterator arch = genArch.begin();
+	   arch != genArch.end(); ++arch)
+      {
+      // Get a pointer to the package structure
       pkgCache::PkgIterator Pkg;
-      if (NewPackage(Pkg,PackageName) == false)
+      Dynamic<pkgCache::PkgIterator> DynPkg(Pkg);
+      if (NewPackage(Pkg, PackageName, *arch) == false)
 	 return _error->Error(_("Error occurred while processing %s (NewPackage)"),PackageName.c_str());
       Counter++;
       if (Counter % 100 == 0 && Progress != 0)
@@ -128,18 +209,20 @@ bool pkgCacheGenerator::MergeList(ListParser &List,
 	 // we first process the package, then the descriptions
 	 // (this has the bonus that we get MMap error when we run out
 	 //  of MMap space)
-	 if (List.UsePackage(Pkg,pkgCache::VerIterator(Cache)) == false)
+	 pkgCache::VerIterator Ver(Cache);
+	 Dynamic<pkgCache::VerIterator> DynVer(Ver);
+	 if (List.UsePackage(Pkg, Ver) == false)
 	    return _error->Error(_("Error occurred while processing %s (UsePackage1)"),
 				 PackageName.c_str());
 
  	 // Find the right version to write the description
  	 MD5SumValue CurMd5 = List.Description_md5();
- 	 pkgCache::VerIterator Ver = Pkg.VersionList();
- 	 map_ptrloc *LastVer = &Pkg->VersionList;
+ 	 Ver = Pkg.VersionList();
 
-	 for (; Ver.end() == false; LastVer = &Ver->NextVer, Ver++)
+	 for (; Ver.end() == false; ++Ver)
  	 {
  	    pkgCache::DescIterator Desc = Ver.DescriptionList();
+	    Dynamic<pkgCache::DescIterator> DynDesc(Desc);
  	    map_ptrloc *LastDesc = &Ver->DescriptionList;
 	    bool duplicate=false;
 
@@ -159,7 +242,11 @@ bool pkgCacheGenerator::MergeList(ListParser &List,
  	       if (MD5SumValue(Desc.md5()) == CurMd5) 
                {
  		  // Add new description
- 		  *LastDesc = NewDescription(Desc, List.DescriptionLanguage(), CurMd5, *LastDesc);
+		  void const * const oldMap = Map.Data();
+		  map_ptrloc const descindex = NewDescription(Desc, List.DescriptionLanguage(), CurMd5, *LastDesc);
+		  if (oldMap != Map.Data())
+		     LastDesc += (map_ptrloc*) Map.Data() - (map_ptrloc*) oldMap;
+		  *LastDesc = descindex;
  		  Desc->ParentPkg = Pkg.Index();
 		  
 		  if ((*LastDesc == 0 && _error->PendingError()) || NewFileDesc(Desc,List) == false)
@@ -173,19 +260,26 @@ bool pkgCacheGenerator::MergeList(ListParser &List,
       }
 
       pkgCache::VerIterator Ver = Pkg.VersionList();
+      Dynamic<pkgCache::VerIterator> DynVer(Ver);
       map_ptrloc *LastVer = &Pkg->VersionList;
+      void const * oldMap = Map.Data();
       int Res = 1;
+      unsigned long const Hash = List.VersionHash();
       for (; Ver.end() == false; LastVer = &Ver->NextVer, Ver++)
       {
 	 Res = Cache.VS->CmpVersion(Version,Ver.VerStr());
-	 if (Res >= 0)
+	 // Version is higher as current version - insert here
+	 if (Res > 0)
+	    break;
+	 // Versionstrings are equal - is hash also equal?
+	 if (Res == 0 && Ver->Hash == Hash)
 	    break;
+	 // proceed with the next till we have either the right
+	 // or we found another version (which will be lower)
       }
-      
-      /* We already have a version for this item, record that we
-         saw it */
-      unsigned long Hash = List.VersionHash();
-      if (Res == 0 && Ver->Hash == Hash)
+
+      /* We already have a version for this item, record that we saw it */
+      if (Res == 0 && Ver.end() == false && Ver->Hash == Hash)
       {
 	 if (List.UsePackage(Pkg,Ver) == false)
 	    return _error->Error(_("Error occurred while processing %s (UsePackage2)"),
@@ -204,35 +298,31 @@ bool pkgCacheGenerator::MergeList(ListParser &List,
 	 }
 	 
 	 continue;
-      }      
-
-      // Skip to the end of the same version set.
-      if (Res == 0)
-      {
-	 for (; Ver.end() == false; LastVer = &Ver->NextVer, Ver++)
-	 {
-	    Res = Cache.VS->CmpVersion(Version,Ver.VerStr());
-	    if (Res != 0)
-	       break;
-	 }
       }
 
       // Add a new version
-      *LastVer = NewVersion(Ver,Version,*LastVer);
+      map_ptrloc const verindex = NewVersion(Ver,Version,*LastVer);
+      if (verindex == 0 && _error->PendingError())
+	 return _error->Error(_("Error occurred while processing %s (NewVersion%d)"),
+			      PackageName.c_str(), 1);
+
+      if (oldMap != Map.Data())
+	 LastVer += (map_ptrloc*) Map.Data() - (map_ptrloc*) oldMap;
+      *LastVer = verindex;
       Ver->ParentPkg = Pkg.Index();
       Ver->Hash = Hash;
 
-      if ((*LastVer == 0 && _error->PendingError()) || List.NewVersion(Ver) == false)
-	 return _error->Error(_("Error occurred while processing %s (NewVersion1)"),
-			      PackageName.c_str());
+      if (List.NewVersion(Ver) == false)
+	 return _error->Error(_("Error occurred while processing %s (NewVersion%d)"),
+			      PackageName.c_str(), 2);
 
       if (List.UsePackage(Pkg,Ver) == false)
 	 return _error->Error(_("Error occurred while processing %s (UsePackage3)"),
 			      PackageName.c_str());
       
       if (NewFileVer(Ver,List) == false)
-	 return _error->Error(_("Error occurred while processing %s (NewVersion2)"),
-			      PackageName.c_str());
+	 return _error->Error(_("Error occurred while processing %s (NewVersion%d)"),
+			      PackageName.c_str(), 3);
 
       // Read only a single record and return
       if (OutVer != 0)
@@ -245,17 +335,23 @@ bool pkgCacheGenerator::MergeList(ListParser &List,
       /* Record the Description data. Description data always exist in
 	 Packages and Translation-* files. */
       pkgCache::DescIterator Desc = Ver.DescriptionList();
+      Dynamic<pkgCache::DescIterator> DynDesc(Desc);
       map_ptrloc *LastDesc = &Ver->DescriptionList;
-      
+
       // Skip to the end of description set
       for (; Desc.end() == false; LastDesc = &Desc->NextDesc, Desc++);
 
       // Add new description
-      *LastDesc = NewDescription(Desc, List.DescriptionLanguage(), List.Description_md5(), *LastDesc);
+      oldMap = Map.Data();
+      map_ptrloc const descindex = NewDescription(Desc, List.DescriptionLanguage(), List.Description_md5(), *LastDesc);
+      if (oldMap != Map.Data())
+	 LastDesc += (map_ptrloc*) Map.Data() - (map_ptrloc*) oldMap;
+      *LastDesc = descindex;
       Desc->ParentPkg = Pkg.Index();
 
       if ((*LastDesc == 0 && _error->PendingError()) || NewFileDesc(Desc,List) == false)
 	 return _error->Error(_("Error occurred while processing %s (NewFileDesc2)"),PackageName.c_str());
+      }
    }
 
    FoundFileDeps |= List.HasFileDeps();
@@ -297,6 +393,7 @@ bool pkgCacheGenerator::MergeFileProvides(ListParser &List)
 	 continue;
       
       pkgCache::PkgIterator Pkg = Cache.FindPkg(PackageName);
+      Dynamic<pkgCache::PkgIterator> DynPkg(Pkg);
       if (Pkg.end() == true)
 	 return _error->Error(_("Error occurred while processing %s (FindPkg)"),
 				PackageName.c_str());
@@ -306,6 +403,7 @@ bool pkgCacheGenerator::MergeFileProvides(ListParser &List)
 
       unsigned long Hash = List.VersionHash();
       pkgCache::VerIterator Ver = Pkg.VersionList();
+      Dynamic<pkgCache::VerIterator> DynVer(Ver);
       for (; Ver.end() == false; Ver++)
       {
 	 if (Ver->Hash == Hash && Version.c_str() == Ver.VerStr())
@@ -323,33 +421,82 @@ bool pkgCacheGenerator::MergeFileProvides(ListParser &List)
    return true;
 }
 									/*}}}*/
-// CacheGenerator::NewPackage - Add a new package			/*{{{*/
+// CacheGenerator::NewGroup - Add a new group				/*{{{*/
 // ---------------------------------------------------------------------
-/* This creates a new package structure and adds it to the hash table */
-bool pkgCacheGenerator::NewPackage(pkgCache::PkgIterator &Pkg,const string &Name)
+/* This creates a new group structure and adds it to the hash table */
+bool pkgCacheGenerator::NewGroup(pkgCache::GrpIterator &Grp, const string &Name)
 {
-   Pkg = Cache.FindPkg(Name);
-   if (Pkg.end() == false)
+   Grp = Cache.FindGrp(Name);
+   if (Grp.end() == false)
       return true;
 
    // Get a structure
-   unsigned long Package = Map.Allocate(sizeof(pkgCache::Package));
-   if (Package == 0)
+   map_ptrloc const Group = AllocateInMap(sizeof(pkgCache::Group));
+   if (unlikely(Group == 0))
       return false;
-   
-   Pkg = pkgCache::PkgIterator(Cache,Cache.PkgP + Package);
-   
+
+   Grp = pkgCache::GrpIterator(Cache, Cache.GrpP + Group);
+   map_ptrloc const idxName = WriteStringInMap(Name);
+   if (unlikely(idxName == 0))
+      return false;
+   Grp->Name = idxName;
+
    // Insert it into the hash table
-   unsigned long Hash = Cache.Hash(Name);
-   Pkg->NextPackage = Cache.HeaderP->HashTable[Hash];
-   Cache.HeaderP->HashTable[Hash] = Package;
-   
-   // Set the name and the ID
-   Pkg->Name = Map.WriteString(Name);
-   if (Pkg->Name == 0)
+   unsigned long const Hash = Cache.Hash(Name);
+   Grp->Next = Cache.HeaderP->GrpHashTable[Hash];
+   Cache.HeaderP->GrpHashTable[Hash] = Group;
+
+   Grp->ID = Cache.HeaderP->GroupCount++;
+   return true;
+}
+									/*}}}*/
+// CacheGenerator::NewPackage - Add a new package			/*{{{*/
+// ---------------------------------------------------------------------
+/* This creates a new package structure and adds it to the hash table */
+bool pkgCacheGenerator::NewPackage(pkgCache::PkgIterator &Pkg,const string &Name,
+					const string &Arch) {
+   pkgCache::GrpIterator Grp;
+   Dynamic<pkgCache::GrpIterator> DynGrp(Grp);
+   if (unlikely(NewGroup(Grp, Name) == false))
+      return false;
+
+   Pkg = Grp.FindPkg(Arch);
+      if (Pkg.end() == false)
+	 return true;
+
+   // Get a structure
+   map_ptrloc const Package = AllocateInMap(sizeof(pkgCache::Package));
+   if (unlikely(Package == 0))
       return false;
+   Pkg = pkgCache::PkgIterator(Cache,Cache.PkgP + Package);
+
+   // Insert the package into our package list
+   if (Grp->FirstPackage == 0) // the group is new
+   {
+      // Insert it into the hash table
+      unsigned long const Hash = Cache.Hash(Name);
+      Pkg->NextPackage = Cache.HeaderP->PkgHashTable[Hash];
+      Cache.HeaderP->PkgHashTable[Hash] = Package;
+      Grp->FirstPackage = Package;
+   }
+   else // Group the Packages together
+   {
+      // this package is the new last package
+      pkgCache::PkgIterator LastPkg(Cache, Cache.PkgP + Grp->LastPackage);
+      Pkg->NextPackage = LastPkg->NextPackage;
+      LastPkg->NextPackage = Package;
+   }
+   Grp->LastPackage = Package;
+
+   // Set the name, arch and the ID
+   Pkg->Name = Grp->Name;
+   Pkg->Group = Grp.Index();
+   map_ptrloc const idxArch = WriteUniqString(Arch.c_str());
+   if (unlikely(idxArch == 0))
+      return false;
+   Pkg->Arch = idxArch;
    Pkg->ID = Cache.HeaderP->PackageCount++;
-   
+
    return true;
 }
 									/*}}}*/
@@ -363,7 +510,7 @@ bool pkgCacheGenerator::NewFileVer(pkgCache::VerIterator &Ver,
       return true;
    
    // Get a structure
-   unsigned long VerFile = Map.Allocate(sizeof(pkgCache::VerFile));
+   map_ptrloc const VerFile = AllocateInMap(sizeof(pkgCache::VerFile));
    if (VerFile == 0)
       return 0;
    
@@ -394,7 +541,7 @@ unsigned long pkgCacheGenerator::NewVersion(pkgCache::VerIterator &Ver,
 					    unsigned long Next)
 {
    // Get a structure
-   unsigned long Version = Map.Allocate(sizeof(pkgCache::Version));
+   map_ptrloc const Version = AllocateInMap(sizeof(pkgCache::Version));
    if (Version == 0)
       return 0;
    
@@ -402,9 +549,10 @@ unsigned long pkgCacheGenerator::NewVersion(pkgCache::VerIterator &Ver,
    Ver = pkgCache::VerIterator(Cache,Cache.VerP + Version);
    Ver->NextVer = Next;
    Ver->ID = Cache.HeaderP->VersionCount++;
-   Ver->VerStr = Map.WriteString(VerStr);
-   if (Ver->VerStr == 0)
+   map_ptrloc const idxVerStr = WriteStringInMap(VerStr);
+   if (unlikely(idxVerStr == 0))
       return 0;
+   Ver->VerStr = idxVerStr;
    
    return Version;
 }
@@ -419,7 +567,7 @@ bool pkgCacheGenerator::NewFileDesc(pkgCache::DescIterator &Desc,
       return true;
    
    // Get a structure
-   unsigned long DescFile = Map.Allocate(sizeof(pkgCache::DescFile));
+   map_ptrloc const DescFile = AllocateInMap(sizeof(pkgCache::DescFile));
    if (DescFile == 0)
       return false;
 
@@ -452,7 +600,7 @@ map_ptrloc pkgCacheGenerator::NewDescription(pkgCache::DescIterator &Desc,
 					    map_ptrloc Next)
 {
    // Get a structure
-   map_ptrloc Description = Map.Allocate(sizeof(pkgCache::Description));
+   map_ptrloc const Description = AllocateInMap(sizeof(pkgCache::Description));
    if (Description == 0)
       return 0;
 
@@ -460,73 +608,149 @@ map_ptrloc pkgCacheGenerator::NewDescription(pkgCache::DescIterator &Desc,
    Desc = pkgCache::DescIterator(Cache,Cache.DescP + Description);
    Desc->NextDesc = Next;
    Desc->ID = Cache.HeaderP->DescriptionCount++;
-   Desc->language_code = Map.WriteString(Lang);
-   Desc->md5sum = Map.WriteString(md5sum.Value());
-   if (Desc->language_code == 0 || Desc->md5sum == 0)
+   map_ptrloc const idxlanguage_code = WriteStringInMap(Lang);
+   map_ptrloc const idxmd5sum = WriteStringInMap(md5sum.Value());
+   if (unlikely(idxlanguage_code == 0 || idxmd5sum == 0))
       return 0;
+   Desc->language_code = idxlanguage_code;
+   Desc->md5sum = idxmd5sum;
 
    return Description;
 }
 									/*}}}*/
-// ListParser::NewDepends - Create a dependency element			/*{{{*/
+// CacheGenerator::FinishCache - do various finish operations		/*{{{*/
+// ---------------------------------------------------------------------
+/* This prepares the Cache for delivery */
+bool pkgCacheGenerator::FinishCache(OpProgress *Progress)
+{
+   // FIXME: add progress reporting for this operation
+   // Do we have different architectures in your groups ?
+   vector<string> archs = APT::Configuration::getArchitectures();
+   if (archs.size() > 1)
+   {
+      // Create Conflicts in between the group
+      pkgCache::GrpIterator G = GetCache().GrpBegin();
+      Dynamic<pkgCache::GrpIterator> DynG(G);
+      for (; G.end() != true; G++)
+      {
+	 string const PkgName = G.Name();
+	 pkgCache::PkgIterator P = G.PackageList();
+	 Dynamic<pkgCache::PkgIterator> DynP(P);
+	 for (; P.end() != true; P = G.NextPkg(P))
+	 {
+	    if (strcmp(P.Arch(),"all") == 0)
+	       continue;
+	    pkgCache::PkgIterator allPkg;
+	    Dynamic<pkgCache::PkgIterator> DynallPkg(allPkg);
+	    pkgCache::VerIterator V = P.VersionList();
+	    Dynamic<pkgCache::VerIterator> DynV(V);
+	    for (; V.end() != true; V++)
+	    {
+	       string const Arch = V.Arch(true);
+	       map_ptrloc *OldDepLast = NULL;
+	       /* MultiArch handling introduces a lot of implicit Dependencies:
+		- MultiArch: same → Co-Installable if they have the same version
+		- Architecture: all → Need to be Co-Installable for internal reasons
+		- All others conflict with all other group members */
+	       bool const coInstall = (V->MultiArch == pkgCache::Version::All ||
+					V->MultiArch == pkgCache::Version::Same);
+	       if (V->MultiArch == pkgCache::Version::All && allPkg.end() == true)
+		  allPkg = G.FindPkg("all");
+	       for (vector<string>::const_iterator A = archs.begin(); A != archs.end(); ++A)
+	       {
+		  if (*A == Arch)
+		     continue;
+		  /* We allow only one installed arch at the time
+		     per group, therefore each group member conflicts
+		     with all other group members */
+		  pkgCache::PkgIterator D = G.FindPkg(*A);
+		  Dynamic<pkgCache::PkgIterator> DynD(D);
+		  if (D.end() == true)
+		     continue;
+		  if (coInstall == true)
+		  {
+		     // Replaces: ${self}:other ( << ${binary:Version})
+		     NewDepends(D, V, V.VerStr(),
+				pkgCache::Dep::Less, pkgCache::Dep::Replaces,
+				OldDepLast);
+		     // Breaks: ${self}:other (!= ${binary:Version})
+		     NewDepends(D, V, V.VerStr(),
+				pkgCache::Dep::NotEquals, pkgCache::Dep::DpkgBreaks,
+				OldDepLast);
+		     if (V->MultiArch == pkgCache::Version::All)
+		     {
+			// Depend on ${self}:all which does depend on nothing
+			NewDepends(allPkg, V, V.VerStr(),
+				   pkgCache::Dep::Equals, pkgCache::Dep::Depends,
+				   OldDepLast);
+		     }
+		  } else {
+			// Conflicts: ${self}:other
+			NewDepends(D, V, "",
+				   pkgCache::Dep::NoOp, pkgCache::Dep::Conflicts,
+				   OldDepLast);
+		  }
+	       }
+	    }
+	 }
+      }
+   }
+   return true;
+}
+									/*}}}*/
+// CacheGenerator::NewDepends - Create a dependency element		/*{{{*/
 // ---------------------------------------------------------------------
 /* This creates a dependency element in the tree. It is linked to the
    version and to the package that it is pointing to. */
-bool pkgCacheGenerator::ListParser::NewDepends(pkgCache::VerIterator Ver,
-					       const string &PackageName,
-					       const string &Version,
-					       unsigned int Op,
-					       unsigned int Type)
+bool pkgCacheGenerator::NewDepends(pkgCache::PkgIterator &Pkg,
+				   pkgCache::VerIterator &Ver,
+				   string const &Version,
+				   unsigned int const &Op,
+				   unsigned int const &Type,
+				   map_ptrloc *OldDepLast)
 {
-   pkgCache &Cache = Owner->Cache;
-   
+   void const * const oldMap = Map.Data();
    // Get a structure
-   unsigned long Dependency = Owner->Map.Allocate(sizeof(pkgCache::Dependency));
-   if (Dependency == 0)
+   map_ptrloc const Dependency = AllocateInMap(sizeof(pkgCache::Dependency));
+   if (unlikely(Dependency == 0))
       return false;
    
    // Fill it in
    pkgCache::DepIterator Dep(Cache,Cache.DepP + Dependency);
+   Dynamic<pkgCache::DepIterator> DynDep(Dep);
    Dep->ParentVer = Ver.Index();
    Dep->Type = Type;
    Dep->CompareOp = Op;
    Dep->ID = Cache.HeaderP->DependsCount++;
-   
-   // Locate the target package
-   pkgCache::PkgIterator Pkg;
-   if (Owner->NewPackage(Pkg,PackageName) == false)
-      return false;
-   
+
    // Probe the reverse dependency list for a version string that matches
    if (Version.empty() == false)
    {
 /*      for (pkgCache::DepIterator I = Pkg.RevDependsList(); I.end() == false; I++)
 	 if (I->Version != 0 && I.TargetVer() == Version)
 	    Dep->Version = I->Version;*/
-      if (Dep->Version == 0)
-	 if ((Dep->Version = WriteString(Version)) == 0)
+      if (Dep->Version == 0) {
+	 map_ptrloc const index = WriteStringInMap(Version);
+	 if (unlikely(index == 0))
 	    return false;
+	 Dep->Version = index;
+      }
    }
-      
+
    // Link it to the package
    Dep->Package = Pkg.Index();
    Dep->NextRevDepends = Pkg->RevDepends;
    Pkg->RevDepends = Dep.Index();
-   
-   /* Link it to the version (at the end of the list)
-      Caching the old end point speeds up generation substantially */
-   if (OldDepVer != Ver)
+
+   // Do we know where to link the Dependency to?
+   if (OldDepLast == NULL)
    {
       OldDepLast = &Ver->DependsList;
       for (pkgCache::DepIterator D = Ver.DependsList(); D.end() == false; D++)
 	 OldDepLast = &D->NextDepends;
-      OldDepVer = Ver;
-   }
+   } else if (oldMap != Map.Data())
+      OldDepLast += (map_ptrloc*) Map.Data() - (map_ptrloc*) oldMap;
 
-   // Is it a file dependency?
-   if (PackageName[0] == '/')
-      FoundFileDeps = true;
-   
    Dep->NextDepends = *OldDepLast;
    *OldDepLast = Dep.Index();
    OldDepLast = &Dep->NextDepends;
@@ -534,36 +758,76 @@ bool pkgCacheGenerator::ListParser::NewDepends(pkgCache::VerIterator Ver,
    return true;
 }
 									/*}}}*/
+// ListParser::NewDepends - Create the environment for a new dependency	/*{{{*/
+// ---------------------------------------------------------------------
+/* This creates a Group and the Package to link this dependency to if
+   needed and handles also the caching of the old endpoint */
+bool pkgCacheGenerator::ListParser::NewDepends(pkgCache::VerIterator &Ver,
+					       const string &PackageName,
+					       const string &Arch,
+					       const string &Version,
+					       unsigned int Op,
+					       unsigned int Type)
+{
+   pkgCache::GrpIterator Grp;
+   Dynamic<pkgCache::GrpIterator> DynGrp(Grp);
+   if (unlikely(Owner->NewGroup(Grp, PackageName) == false))
+      return false;
+
+   // Locate the target package
+   pkgCache::PkgIterator Pkg = Grp.FindPkg(Arch);
+   Dynamic<pkgCache::PkgIterator> DynPkg(Pkg);
+   if (Pkg.end() == true) {
+      if (unlikely(Owner->NewPackage(Pkg, PackageName, Arch) == false))
+	 return false;
+   }
+
+   // Is it a file dependency?
+   if (unlikely(PackageName[0] == '/'))
+      FoundFileDeps = true;
+
+   /* Caching the old end point speeds up generation substantially */
+   if (OldDepVer != Ver) {
+      OldDepLast = NULL;
+      OldDepVer = Ver;
+   }
+
+   return Owner->NewDepends(Pkg, Ver, Version, Op, Type, OldDepLast);
+}
+									/*}}}*/
 // ListParser::NewProvides - Create a Provides element			/*{{{*/
 // ---------------------------------------------------------------------
 /* */
-bool pkgCacheGenerator::ListParser::NewProvides(pkgCache::VerIterator Ver,
-					        const string &PackageName,
+bool pkgCacheGenerator::ListParser::NewProvides(pkgCache::VerIterator &Ver,
+					        const string &PkgName,
+						const string &PkgArch,
 						const string &Version)
 {
    pkgCache &Cache = Owner->Cache;
 
    // We do not add self referencing provides
-   if (Ver.ParentPkg().Name() == PackageName)
+   if (Ver.ParentPkg().Name() == PkgName && PkgArch == Ver.Arch(true))
       return true;
    
    // Get a structure
-   unsigned long Provides = Owner->Map.Allocate(sizeof(pkgCache::Provides));
-   if (Provides == 0)
+   map_ptrloc const Provides = Owner->AllocateInMap(sizeof(pkgCache::Provides));
+   if (unlikely(Provides == 0))
       return false;
    Cache.HeaderP->ProvidesCount++;
    
    // Fill it in
    pkgCache::PrvIterator Prv(Cache,Cache.ProvideP + Provides,Cache.PkgP);
+   Dynamic<pkgCache::PrvIterator> DynPrv(Prv);
    Prv->Version = Ver.Index();
    Prv->NextPkgProv = Ver->ProvidesList;
    Ver->ProvidesList = Prv.Index();
-   if (Version.empty() == false && (Prv->ProvideVersion = WriteString(Version)) == 0)
+   if (Version.empty() == false && unlikely((Prv->ProvideVersion = WriteString(Version)) == 0))
       return false;
    
    // Locate the target package
    pkgCache::PkgIterator Pkg;
-   if (Owner->NewPackage(Pkg,PackageName) == false)
+   Dynamic<pkgCache::PkgIterator> DynPkg(Pkg);
+   if (unlikely(Owner->NewPackage(Pkg,PkgName, PkgArch) == false))
       return false;
    
    // Link it to the package
@@ -583,24 +847,29 @@ bool pkgCacheGenerator::SelectFile(const string &File,const string &Site,
 				   unsigned long Flags)
 {
    // Get some space for the structure
-   CurrentFile = Cache.PkgFileP + Map.Allocate(sizeof(*CurrentFile));
-   if (CurrentFile == Cache.PkgFileP)
+   map_ptrloc const idxFile = AllocateInMap(sizeof(*CurrentFile));
+   if (unlikely(idxFile == 0))
       return false;
-   
+   CurrentFile = Cache.PkgFileP + idxFile;
+
    // Fill it in
-   CurrentFile->FileName = Map.WriteString(File);
-   CurrentFile->Site = WriteUniqString(Site);
+   map_ptrloc const idxFileName = WriteStringInMap(File);
+   map_ptrloc const idxSite = WriteUniqString(Site);
+   if (unlikely(idxFileName == 0 || idxSite == 0))
+      return false;
+   CurrentFile->FileName = idxFileName;
+   CurrentFile->Site = idxSite;
    CurrentFile->NextFile = Cache.HeaderP->FileList;
    CurrentFile->Flags = Flags;
    CurrentFile->ID = Cache.HeaderP->PackageFileCount;
-   CurrentFile->IndexType = WriteUniqString(Index.GetType()->Label);
+   map_ptrloc const idxIndexType = WriteUniqString(Index.GetType()->Label);
+   if (unlikely(idxIndexType == 0))
+      return false;
+   CurrentFile->IndexType = idxIndexType;
    PkgFileName = File;
    Cache.HeaderP->FileList = CurrentFile - Cache.PkgFileP;
    Cache.HeaderP->PackageFileCount++;
 
-   if (CurrentFile->FileName == 0)
-      return false;
-   
    if (Progress != 0)
       Progress->SubProgress(Index.Size());
    return true;
@@ -640,18 +909,25 @@ unsigned long pkgCacheGenerator::WriteUniqString(const char *S,
    }
    
    // Get a structure
-   unsigned long Item = Map.Allocate(sizeof(pkgCache::StringItem));
+   void const * const oldMap = Map.Data();
+   map_ptrloc const Item = AllocateInMap(sizeof(pkgCache::StringItem));
    if (Item == 0)
       return 0;
 
+   map_ptrloc const idxString = WriteStringInMap(S,Size);
+   if (unlikely(idxString == 0))
+      return 0;
+   if (oldMap != Map.Data()) {
+      Last += (map_ptrloc*) Map.Data() - (map_ptrloc*) oldMap;
+      I += (pkgCache::StringItem*) Map.Data() - (pkgCache::StringItem*) oldMap;
+   }
+   *Last = Item;
+
    // Fill in the structure
    pkgCache::StringItem *ItemP = Cache.StringItemP + Item;
    ItemP->NextItem = I - Cache.StringItemP;
-   *Last = Item;
-   ItemP->String = Map.WriteString(S,Size);
-   if (ItemP->String == 0)
-      return 0;
-   
+   ItemP->String = idxString;
+
    Bucket = ItemP;
    return ItemP->String;
 }
@@ -769,7 +1045,7 @@ static unsigned long ComputeSize(FileIterator Start,FileIterator End)
 // ---------------------------------------------------------------------
 /* */
 static bool BuildCache(pkgCacheGenerator &Gen,
-		       OpProgress &Progress,
+		       OpProgress *Progress,
 		       unsigned long &CurrentSize,unsigned long TotalSize,
 		       FileIterator Start, FileIterator End)
 {
@@ -790,7 +1066,8 @@ static bool BuildCache(pkgCacheGenerator &Gen,
       }
       
       unsigned long Size = (*I)->Size();
-      Progress.OverallProgress(CurrentSize,TotalSize,Size,_("Reading package lists"));
+      if (Progress != NULL)
+	 Progress->OverallProgress(CurrentSize,TotalSize,Size,_("Reading package lists"));
       CurrentSize += Size;
       
       if ((*I)->Merge(Gen,Progress) == false)
@@ -799,13 +1076,15 @@ static bool BuildCache(pkgCacheGenerator &Gen,
 
    if (Gen.HasFileDeps() == true)
    {
-      Progress.Done();
+      if (Progress != NULL)
+	 Progress->Done();
       TotalSize = ComputeSize(Start, End);
       CurrentSize = 0;
       for (I = Start; I != End; I++)
       {
 	 unsigned long Size = (*I)->Size();
-	 Progress.OverallProgress(CurrentSize,TotalSize,Size,_("Collecting File Provides"));
+	 if (Progress != NULL)
+	    Progress->OverallProgress(CurrentSize,TotalSize,Size,_("Collecting File Provides"));
 	 CurrentSize += Size;
 	 if ((*I)->MergeFileProvides(Gen,Progress) == false)
 	    return false;
@@ -815,7 +1094,21 @@ static bool BuildCache(pkgCacheGenerator &Gen,
    return true;
 }
 									/*}}}*/
-// MakeStatusCache - Construct the status cache				/*{{{*/
+// CacheGenerator::CreateDynamicMMap - load an mmap with configuration options	/*{{{*/
+DynamicMMap* pkgCacheGenerator::CreateDynamicMMap(FileFd *CacheF, unsigned long Flags) {
+   unsigned long const MapStart = _config->FindI("APT::Cache-Start", 24*1024*1024);
+   unsigned long const MapGrow = _config->FindI("APT::Cache-Grow", 1*1024*1024);
+   unsigned long const MapLimit = _config->FindI("APT::Cache-Limit", 0);
+   Flags |= MMap::Moveable;
+   if (_config->FindB("APT::Cache-Fallback", false) == true)
+      Flags |= MMap::Fallback;
+   if (CacheF != NULL)
+      return new DynamicMMap(*CacheF, Flags, MapStart, MapGrow, MapLimit);
+   else
+      return new DynamicMMap(Flags, MapStart, MapGrow, MapLimit);
+}
+									/*}}}*/
+// CacheGenerator::MakeStatusCache - Construct the status cache		/*{{{*/
 // ---------------------------------------------------------------------
 /* This makes sure that the status cache (the cache that has all 
    index files from the sources list and all local ones) is ready
@@ -823,11 +1116,13 @@ static bool BuildCache(pkgCacheGenerator &Gen,
    the cache will be stored there. This is pretty much mandetory if you
    are using AllowMem. AllowMem lets the function be run as non-root
    where it builds the cache 'fast' into a memory buffer. */
-bool pkgMakeStatusCache(pkgSourceList &List,OpProgress &Progress,
+__deprecated bool pkgMakeStatusCache(pkgSourceList &List,OpProgress &Progress,
+			MMap **OutMap, bool AllowMem)
+   { return pkgCacheGenerator::MakeStatusCache(List, &Progress, OutMap, AllowMem); }
+bool pkgCacheGenerator::MakeStatusCache(pkgSourceList &List,OpProgress *Progress,
 			MMap **OutMap,bool AllowMem)
 {
    bool const Debug = _config->FindB("Debug::pkgCacheGen", false);
-   unsigned long const MapSize = _config->FindI("APT::Cache-Limit",24*1024*1024);
    
    vector<pkgIndexFile *> Files;
    for (vector<metaIndex *>::const_iterator i = List.begin();
@@ -848,7 +1143,20 @@ bool pkgMakeStatusCache(pkgSourceList &List,OpProgress &Progress,
    // Decide if we can write to the files..
    string const CacheFile = _config->FindFile("Dir::Cache::pkgcache");
    string const SrcCacheFile = _config->FindFile("Dir::Cache::srcpkgcache");
-   
+
+   // ensure the cache directory exists
+   if (CacheFile.empty() == false || SrcCacheFile.empty() == false)
+   {
+      string dir = _config->FindDir("Dir::Cache");
+      size_t const len = dir.size();
+      if (len > 5 && dir.find("/apt/", len - 6, 5) == len - 5)
+	 dir = dir.substr(0, len - 5);
+      if (CacheFile.empty() == false)
+	 CreateDirectory(dir, flNotFile(CacheFile));
+      if (SrcCacheFile.empty() == false)
+	 CreateDirectory(dir, flNotFile(SrcCacheFile));
+   }
+
    // Decide if we can write to the cache
    bool Writeable = false;
    if (CacheFile.empty() == false)
@@ -861,13 +1169,15 @@ bool pkgMakeStatusCache(pkgSourceList &List,OpProgress &Progress,
 
    if (Writeable == false && AllowMem == false && CacheFile.empty() == false)
       return _error->Error(_("Unable to write to %s"),flNotFile(CacheFile).c_str());
-   
-   Progress.OverallProgress(0,1,1,_("Reading package lists"));
-   
+
+   if (Progress != NULL)
+      Progress->OverallProgress(0,1,1,_("Reading package lists"));
+
    // Cache is OK, Fin.
    if (CheckValidity(CacheFile,Files.begin(),Files.end(),OutMap) == true)
    {
-      Progress.OverallProgress(1,1,1,_("Reading package lists"));
+      if (Progress != NULL)
+	 Progress->OverallProgress(1,1,1,_("Reading package lists"));
       if (Debug == true)
 	 std::clog << "pkgcache.bin is valid - no need to build anything" << std::endl;
       return true;
@@ -884,7 +1194,7 @@ bool pkgMakeStatusCache(pkgSourceList &List,OpProgress &Progress,
       unlink(CacheFile.c_str());
       CacheF = new FileFd(CacheFile,FileFd::WriteEmpty);
       fchmod(CacheF->Fd(),0644);
-      Map = new DynamicMMap(*CacheF,MMap::Public,MapSize);
+      Map = CreateDynamicMMap(CacheF, MMap::Public);
       if (_error->PendingError() == true)
 	 return false;
       if (Debug == true)
@@ -893,7 +1203,7 @@ bool pkgMakeStatusCache(pkgSourceList &List,OpProgress &Progress,
    else
    {
       // Just build it in memory..
-      Map = new DynamicMMap(0,MapSize);
+      Map = CreateDynamicMMap(NULL);
       if (Debug == true)
 	 std::clog << "Open memory Map (not filebased)" << std::endl;
    }
@@ -917,12 +1227,15 @@ bool pkgMakeStatusCache(pkgSourceList &List,OpProgress &Progress,
       TotalSize = ComputeSize(Files.begin()+EndOfSource,Files.end());
 
       // Build the status cache
-      pkgCacheGenerator Gen(Map.Get(),&Progress);
+      pkgCacheGenerator Gen(Map.Get(),Progress);
       if (_error->PendingError() == true)
 	 return false;
       if (BuildCache(Gen,Progress,CurrentSize,TotalSize,
 		     Files.begin()+EndOfSource,Files.end()) == false)
 	 return false;
+
+      // FIXME: move me to a better place
+      Gen.FinishCache(Progress);
    }
    else
    {
@@ -931,7 +1244,7 @@ bool pkgMakeStatusCache(pkgSourceList &List,OpProgress &Progress,
       TotalSize = ComputeSize(Files.begin(),Files.end());
       
       // Build the source cache
-      pkgCacheGenerator Gen(Map.Get(),&Progress);
+      pkgCacheGenerator Gen(Map.Get(),Progress);
       if (_error->PendingError() == true)
 	 return false;
       if (BuildCache(Gen,Progress,CurrentSize,TotalSize,
@@ -965,6 +1278,9 @@ bool pkgMakeStatusCache(pkgSourceList &List,OpProgress &Progress,
       if (BuildCache(Gen,Progress,CurrentSize,TotalSize,
 		     Files.begin()+EndOfSource,Files.end()) == false)
 	 return false;
+
+      // FIXME: move me to a better place
+      Gen.FinishCache(Progress);
    }
    if (Debug == true)
       std::clog << "Caches are ready for shipping" << std::endl;
@@ -987,32 +1303,37 @@ bool pkgMakeStatusCache(pkgSourceList &List,OpProgress &Progress,
    return true;
 }
 									/*}}}*/
-// MakeOnlyStatusCache - Build a cache with just the status files	/*{{{*/
+// CacheGenerator::MakeOnlyStatusCache - Build only a status files cache/*{{{*/
 // ---------------------------------------------------------------------
 /* */
-bool pkgMakeOnlyStatusCache(OpProgress &Progress,DynamicMMap **OutMap)
+__deprecated bool pkgMakeOnlyStatusCache(OpProgress &Progress,DynamicMMap **OutMap)
+   { return pkgCacheGenerator::MakeOnlyStatusCache(&Progress, OutMap); }
+bool pkgCacheGenerator::MakeOnlyStatusCache(OpProgress *Progress,DynamicMMap **OutMap)
 {
-   unsigned long MapSize = _config->FindI("APT::Cache-Limit",20*1024*1024);
    vector<pkgIndexFile *> Files;
    unsigned long EndOfSource = Files.size();
    if (_system->AddStatusFiles(Files) == false)
       return false;
-   
-   SPtr<DynamicMMap> Map = new DynamicMMap(0,MapSize);
+
+   SPtr<DynamicMMap> Map = CreateDynamicMMap(NULL);
    unsigned long CurrentSize = 0;
    unsigned long TotalSize = 0;
    
    TotalSize = ComputeSize(Files.begin()+EndOfSource,Files.end());
    
    // Build the status cache
-   Progress.OverallProgress(0,1,1,_("Reading package lists"));
-   pkgCacheGenerator Gen(Map.Get(),&Progress);
+   if (Progress != NULL)
+      Progress->OverallProgress(0,1,1,_("Reading package lists"));
+   pkgCacheGenerator Gen(Map.Get(),Progress);
    if (_error->PendingError() == true)
       return false;
    if (BuildCache(Gen,Progress,CurrentSize,TotalSize,
 		  Files.begin()+EndOfSource,Files.end()) == false)
       return false;
-   
+
+   // FIXME: move me to a better place
+   Gen.FinishCache(Progress);
+
    if (_error->PendingError() == true)
       return false;
    *OutMap = Map.UnGuard();