* merge of the debian-expermental-ma branch

  
* debian/control:
  - add dependency on zlib-dev for libapt-pkg-dev
* apt-pkg/cacheset.cc:
  - [ABI BREAK] add an ErrorType option to CacheSetHelper
* cmdline/apt-cache.cc:
  - use Notice instead of Error in the CacheSetHelper messages
    for compat reasons. Otherwise tools like sbuild blow up
  - return success in show if a virtual package was given
* debian/control:
  - remove libcurl3-gnutls-dev alternative as the package is gone
  - increase needed version of libcurl4-gnutls-dev to >= 7.19.0
    as we use CURLOPT_{ISSUERCERT,CRLFILE} (Closes: #589642)
* apt-pkg/deb/dpkgpm.cc:
  - Write architecture information to history file.
  - Add to history whether a change was automatic or not.
* apt-pkg/contrib/fileutl.cc:
  - Add FileFd::OpenDescriptor() (needed for python-apt's #383617).
* cmdline/apt-get.cc:
  - Support large filesystems by using statvfs64() instead of statvfs()
    and statfs64() instead of statfs() (Closes: #590513).
* apt-pkg/cdrom.cc:
  - Use link() instead of rename() for creating the CD database backup;
    otherwise there would be a short time without any database.
* apt-pkg/depcache.cc:
  - handle "circular" conflicts for "all" packages correctly
* cmdline/apt-cache.cc:
  - be able to omit dependency types in (r)depends (Closes: #319006)
  - show in (r)depends the canidate per default instead of newest
  - share the (r)depends code instead of codecopy
* apt-pkg/cacheset.cc:
  - move them back to the library as they look stable now
  - add a 'newest' pseudo target release as in pkg/newest
* apt-pkg/pkgcache.cc:
  - prefer non-virtual packages in FindPreferredPkg (Closes: #590041)
* test/integration/*:
  - add with bug#590041 testcase a small test "framework"
* apt-pkg/orderlist.cc:
  - try to install another or-group member in DepRemove before
    breaking the or group (Closes: #590438)
  - configure also the replacement before remove by adding Immediate flag

* apt-pkg/contrib/error.{cc,h}
  - docstring cleanup
  - add inline DumpError() to avoid subtle API break
* apt-pkg/contrib/error.{cc,h}:
  - remove constness of va_list parameter to fix build on amd64 and co
    Thanks Eric Valette! (Closes: #588610)
* apt-pkg/deb/debmetaindex.cc:
  - do not query each architecture for flat file archives
  - fix typo preventing display of architecture in Info()
* methods/bzip2.cc:
  - add a copycat of the old gzip.cc as we need it for bzip2 and lzma
* debian/rules:
  - Make DEB_BUILD_OPTIONS=noopt actually work by passing the right
    CXXFLAGS.
* apt-pkg/contrib/fileutl.{h,cc}:
  - Add support for reading of gzipped files with the new "ReadOnlyGzip"
    OpenMode. (Closes: #188407)
  - Link against zlib (in apt-pkg/makefile) and add zlib build dependency.
  - [ABI BREAK] This adds a new private member to FileFd, but its
    initialization is in the public header file.
* configure.in:
  - Check for zlib library and headers.
* apt-pkg/acquire-item.cc, apt-pkg/deb/debindexfile.cc,
  apt-pkg/deb/debrecords.cc, apt-pkg/deb/debsrcrecords.h,
  cmdline/apt-cache.cc:
  - Open Packages, Sources, and Translations indexes in "ReadOnlyGzip" mode.
* apt-pkg/deb/debindexfile.cc:
  - If we do not find uncompressed package/source/translation indexes, look
    for gzip compressed ones.
* apt-pkg/acquire-item.cc:
  - If the Acquire::GzipIndexes option is true and we download a gzipped
    index file, keep it as it is (and rename to .gz) instead of
    uncompressing it.
* doc/apt.conf.5.xml:
  - Document the new Acquire::GzipIndexes option.
* doc/po/apt-doc.pot, doc/po/de.po:
  - German translation of new Acquire::GzipIndexes option.
* Add test/test-indexes.sh:
  - Test behaviour of index retrieval and usage, in particular with
    uncompressed and gzip compressed indexes.
* methods/gzip.cc: With FileFd now being able to read gzipped files, there
  is no need for the gzip method any more to spawn an external gzip process.
  Rewrite it to use FileFd directly, which makes the code a lot simpler, and
  also using less memory and overhead.
* doc/apt.conf.5.xml:
  - add and document APT::Cache-{Start,Grow,Limit} options for mmap control
* apt-pkg/contrib/fileutl.cc:
  - do not fail von double close()
* cmdline/cacheset.cc:
  - doesn't include it in the library for now as it is too volatile
  - get the candidate either from an already built depcache
    or use the policy which is a bit faster than depcache generation
  - get packages by task^ with FromTask()
  - only print errors if all tries to get a package by string failed
  - factor out code to get a single package FromName()
  - check in Grouped* first without modifier interpretation
* cmdline/apt-get.cc:
  - use the cachsets in the install commands
  - make the specify order of packages irrelevant (Closes: #196021)
* apt-pkg/orderlist.cc:
  - untouched packages are never missing
* apt-pkg/packagemanager.cc:
  - packages that are not touched doesn't need to be unpacked
* debian/control:
  - remove intltool's dependency as it is an ubuntu artefact
* apt-pkg/depcache.cc:
  - SetCandidateVer for all pseudo packages
  - SetReInstall for the "all" package of a pseudo package
  - use the new MatchAgainstConfig for the DefaultRootSetFunc
  - always mark the all package if a pseudo package is marked for install
* apt-pkg/contrib/error.{cc,h}:
  - complete rewrite but use the same API
  - add NOTICE and DEBUG as new types of a message
  - add a simple stack handling to be able to delay error handling
* apt-pkg/aptconfiguration.cc:
  - show a deprecation notice for APT::Acquire::Translation
* apt-pkg/contrib/configuration.{cc,h}:
  - add a wrapper to match strings against configurable regex patterns
* apt-pkg/contrib/fileutl.cc:
  - show notice about ignored file instead of being always silent
  - add a Dir::Ignore-Files-Silently list option to control the notice
* apt-pkg/policy.h:
  - add another round of const& madness as the previous round accidentally
    NOT overrides the virtual GetCandidateVer() method (Closes: #587725)
* apt-pkg/pkgcachegen.{cc,h}:
  - make the used MMap moveable (and therefore dynamic resizeable) by
    applying (some) mad pointer magic (Closes: #195018)
* apt-pkg/deb/dpkgpm.cc:
  - make the apt/term.log output unbuffered (thanks to Matt Zimmerman)
* methods/ftp.h:
  - Handle different logins on the same server (Closes: #586904).
* apt-pkg/deb/deblistparser.cc:
  - Handle architecture wildcards (Closes: #547724).
* apt-pkg/versionmatch.cc:
  - Support matching pins by regular expressions or glob() like patterns,
    regular expressions have to be put between to slashes; for example,
    /.*/.
* apt-pkg/contrib/fileutl.cc:
  - Make FileFd replace files atomically in WriteTemp mode (for cache, etc).
* debian/control:
  - Set Standards-Version to 3.9.0
* apt-pkg/cachefile.h:
  - make pkgPolicy public again, libapt-pkg-perl (and probably
    others) get unhappy without that
* merge the remaining Ubuntu change:
  - on gpg verification failure warn and restore the last known
    good state
  - on failure display the IP of the server (useful for servers
    that use round robin DNS)
  - support Original-Maintainer in RewritePackageOrder
  - enable cdrom autodetection via libudev by default
  - show message about Vcs in use when apt-get source is run for
    packages maintained in a Vcs
  - better support transitional packages with mark auto-installed. 
    when the transitional package is in "oldlibs" the new package
    is not marked auto installed (same is true for section
    metapackages)
  - provide new "deb mirror://archive.foo/mirrors.list sid main"
    method expects a list of mirrors (generated on the server e.g.
    via geoip) and will use that, including cycle on failure
  - write apport crash file on package failure (disabled by default
    on debian until apport is available)
  - support mirror failure reporting (disabled by default on debian)

* apt-pkg/deb/dpkgpm.cc:
  - write Disappeared also to the history.log
  - forward manual-installed bit on package disappearance
* apt-pkg/deb/debsystem.cc:
  - add better config item for extended_states file
* apt-pkg/pkgcache.h:
  - switch {,Install-}Size to unsigned long long
* apt-pkg/depcache.cc:
  - do the autoremove mark process also for required packages to handle
    these illegally depending on lower priority packages (Closes: #583517)
  - try harder to find the other pseudo versions for autoremove multiarch
  - correct "Dangerous iterator usage" pointed out by cppcheck
  - deal with long long, not with int to remove 2GB Limit (LP: #250909)
  - deprecate AddSize with Multiplier as it is unused and switch to
    boolean instead to handle the sizes more gracefully.
  - switch i{Download,Usr}Size from double to (un)signed long long
* apt-pkg/aptconfiguration.cc:
  - remove duplicate architectures in getArchitectures()
* apt-pkg/indexrecords.{cc,h}:
  - backport forgotten Valid-Until patch from the obsolete experimental
    branch to prevent replay attacks better, thanks to Thomas Viehmann
    for the initial patch! (Closes: #499897)
  - add a constant Exists check for MetaKeys
* apt-pkg/acquire-item.cc:
  - do not try PDiff if it is not listed in the Meta file
  - sent Last-Modified header also for Translation files
* apt-pkg/cacheiterator.h:
  - let pkgCache::Iterator inherent std::iterator
* ftparchive/writer.h:
  - add a virtual destructor to FTWScanner class (for cppcheck)
* apt-pkg/cacheset.{cc,h}:
  - add simple wrapper around std::set for cache structures
  - move regex magic from apt-get to new FromRegEx method
  - move cmdline parsing from apt-cache to new FromCommandLine method
  - support special release-modifier 'installed' and 'candidate'
* apt-pkg/contrib/cmdline.cc:
  - fix segfault in SaveInConfig caused by writing over char[] sizes
* apt-pkg/pkgcache.cc:
  - get the best matching arch package from a group with FindPreferredPkg
* cmdline/apt-cache.cc:
  - make the search multiarch compatible by using GrpIterator instead
  - use pkgCacheFile and the new CacheSets all over the place
  - add --target-release option (Closes: #115520)
  - accept pkg/release and pkg=version in show and co. (Closes: #236270)
  - accept package versions in the unmet command
* cmdline/apt-get.cc:
  - use unsigned long long instead of double to store values it gets
* apt-pkg/cachefile.{cc,h}:
  - split Open() into submethods to be able to build only parts
  - make the OpProgress optional in the Cache buildprocess
  - store also the SourceList we use internally for export
* doc/apt.conf.5.xml:
  - document the new Valid-Until related options
* apt-pkg/contrib/strutl.cc:
  - split StrToTime() into HTTP1.1 and FTP date parser methods and
    use strptime() instead of some self-made scanf mangling
  - use the portable timegm shown in his manpage instead of a strange
    looking code copycat from wget
* ftparchive/writer.cc:
  - add ValidTime option to generate a Valid-Until header in Release file
* apt-pkg/policy.cc:
  - get the candidate right for a not-installed pseudo package if
    his non-pseudo friend is installed
* apt-pkg/indexcopy.cc:
  - move the gpg codecopy to a new method and use it also in methods/gpgv.cc
* cmdline/apt-get.cc:
  - rerun dpkg-source in source if --fix-broken is given (Closes: #576752)
  - don't suggest held packages as they are installed (Closes: #578135)
  - handle multiple --{tar,diff,dsc}-only options correctly
  - show at the end of the install process a list of disappeared packages
* cmdline/apt-cache.cc:
  - use GroupCount for package names in stats and add a package struct line
* methods/rred.cc:
  - use the patchfile modification time instead of the one from the
    "old" file - thanks to Philipp Weis for noticing! (Closes: #571541)
* debian/rules:
  - remove targets referring to CVS or arch as they are useless
  - use $(CURDIR) instead of $(pwd)
  - use dpkg-buildflags if available for CXXFLAGS
* README.arch:
  - remove the file completely as it has no use nowadays
* apt-pkg/depcache.cc:
  - be doublesure that the killer query is empty before starting reinstall
* methods/gpgv.cc:
  - remove the keyrings count limit by using vector magic
* contrib/mmap.cc:
  - clarify "MMap reached size limit" error message, thanks Ivan Masár!
* doc/apt.ent
  - add entities for the current oldstable/stable/testing codenames
* doc/sources.list.5.xml:
  - use stable-codename instead of stable in the examples (Closes: #531492)
* doc/apt_preferences.5.xml:
  - adapt some examples here to use current codenames as well
  - add "NotAutomatic: yes" handling, thanks Osamu Aoki (Closes: #490347)
* debian/libapt-pkg-doc.doc-base.cache:
  - remove yet another reference to the removed cache.sgml
* doc/apt-get.8.xml:
  - do not say explicit target_release_{name,version,codename}, it should
    be clear by itself and 'man' can break lines again (Closes: #566166)
  - remove the gnome-apt reference as it is removed from unstable
* apt-pkg/deb/dpkgpm.cc:
  - add 'disappear' to the known processing states, thanks Jonathan Nieder
* apt-pkg/packagemanager.h:
  - export info about disappeared packages with GetDisappearedPackages()
* methods/http.{cc,h}:
  - code cleanup, use enums instead of magic ints

* debian/rules:
  - spell out some less known options to reduce manpage consultation-rate
  - Use POSIX command substitution: $(<command sequence>)
  - Remove EOL whitespace (Closes: #577804)
* apt-pkg/acquire-item.cc:
  - Fix pkgAcqFile::Custom600Headers() to always return something.

* apt-pkg/depcache.cc:
  - rewrite the pseudo package reinstaller to be more intelligent
    in his package choices
* apt-pkg/packagemanager.cc:
  - don't try to "unpack" pseudo packages twice
* apt-pkg/contrib/fileutl.cc:
  - add a parent-guarded "mkdir -p" as CreateDirectory()
* apt-pkg/acquire.{cc,h}:
  - add a delayed constructor with Setup() for success reporting
  - check for and create directories in Setup if needed instead of
    error out unfriendly in the Constructor (Closes: #523920, #525783)
  - optional handle a lock file in Setup()
* apt-pkg/acquire-item.cc:
  - Acquire::ForceHash to force method for expected hash
* cmdline/apt-get.cc:
  - remove the lock file handling and let Acquire take care of it instead
  - display MD5Sum in --print-uris if not forced to use another method
    instead of displaying the strongest available (Closes: #576420)
  - regex for package names executed on Grp- not PkgIterator
  - show non-candidates as fallback for virtual packages (Closes: #578385)
  - set also "all" to this version for pseudo packages in TryToChangeVer
* apt-pkg/deb/dpkgpm.cc:
  - remove Chroot-Directory from files passed to install commands.
    Thanks to Kel Modderman for report & patch! (Closes: #577226)
* ftparchive/writer.cc:
  - remove 999 chars Files and Checksums rewrite limit (Closes: #577759)
* cmdline/apt-cache.cc:
  - align Installed and Candidate Version in policy so they can be compared
    easier, thanks Ralf Gesellensetter for the pointer! (Closes: #578657)
* doc/apt.ent:
  - Add a note about APT_CONFIG in the -c description (Closes: #578267)
* doc/po/de.po:
  - correct typos in german apt_preferences manpage, thanks Chris Leick!
* apt-pkg/sourcelist.cc:
  - be less strict and accept [option=value] as well
* apt-pkg/contrib/configuration.cc:
  - error out if #clear directive has no argument
* doc/files.sgml:
  - sync documentation with status quo, regarding files/directories in
    use, extended_states and uri schemes.
* doc/cache.sgml:
  - drop the file in favor of inplace documentation with doxygen
* apt-pkg/pkgcache.h:
  - enhance the Groups ABI by providing a ID as the other structs does
  - check also the size of the Group struct then checking for the others
* cmdline/apt-get.cc:
  - replace backticks with single quotes around fix-broken command
    in the broken packages message. (Closes: #577168)
* dselect/install:
  - modernize if-statements not to use 'x' (Closes: #577117)
  - replace backticks with POSIX $() (Closes: #577116)
* cmdline/apt-get.cc:
  - fix crash when pkg.VersionList() is empty
  - install html doxygen in libapt-pkg-doc 
* debian/control:
  - build-depend on doxygen
* apt-pkg/contrib/weakptr.h:
  - add a class WeakPointable which allows one to register weak pointers to
    an object which will be set to NULL when the object is deallocated.
* [ABI break] apt-pkg/acquire{-worker,-item,}.h:
  - subclass pkgAcquire::{Worker,Item,ItemDesc} from WeakPointable.
* apt-pkg/pkgcache.cc:
  - Merge fix from David to correct handling in single-arch environments.
* cmdline/apt-cache.cc:
  - Add a showauto command to apt-cache.
* cmdline/apt-get.cc:
  - Add apt-get markauto and unmarkauto commands.
* [BREAK] merge MultiArch-ABI. We don't support MultiArch,
  but we support the usage of the new ABI so libapt users
  can start to prepare for MultiArch (Closes: #536029)
* Ignore :qualifiers after package name in build dependencies
  in the library by default, but try to honour them in apt-get
  as we have some sort of MultiArch support ready (Closes: #558103)
* add translation of the manpages to PT (portuguese)
  Thanks to Américo Monteiro!
* Switch to dpkg-source 3.0 (native) format
* apt-pkg/depcache.cc:
  - remove Auto-Installed information from extended_states
    together with the package itself (Closes: #572364)
* cmdline/apt-mark:
  - don't crash if no arguments are given (Closes: #570962)
* debian/control:
  - remove some years old and obsolete Replaces
  - add automake/conf build-depends/conflicts as recommend by
    the autotools-dev README (Closes: #572615)
* apt-pkg/contrib/mmap.{h,cc}:
  - add char[] fallback for filesystems without shared writable
    mmap() like JFFS2. Thanks to Marius Vollmer for writing
    and to Loïc Minier for pointing to the patch! (Closes: #314334)
* doc/apt_preferences.5.xml:
  - fix two typos and be more verbose in the novice warning.
    Thanks to Osamu Aoki for pointing it out! (Closes: #567669)
  - fix a=sid vs. n=sid typo, thanks Ansgar Burchardt!
  - origin can be used to match a hostname (Closes: #352667)
  - remove wrong pin-priority is optional remark (Closes: #574944)
* apt-pkg/deb/dpkgpm.cc:
  - fix error message construction in OpenLog()
  - if available store the Commandline in the history
* cmdline/apt-get.cc:
  - add a --only-upgrade flag to install command (Closes: #572259)
  - fix memory leaks in error conditions in DoSource()
  - try version match in FindSrc first exact than fuzzy (LP: #551178)
* apt-pkg/contrib/cmndline.cc:
  - save Commandline in Commandline::AsString for logging
* apt-pkg/deb/debversion.cc:
  - consider absent of debian revision equivalent to 0 (Closes: #573592)
* doc/makefile, doc/*:
  - generate subdirectories for building the manpages in on the fly
    depending on the po files we have.
* apt-pkg/pkgcachegen.cc:
  - merge versions correctly even if multiple different versions
    with the same version number are available.
    Thanks to Magnus Holmgren for the patch! (Closes: #351056)
* ftparchive/writer.cc:
  - write LongDescriptions if they shouldn't be included in Packages
    file into i18n/Translation-en by default.
* doc/po/de.po:
  - correct a few typos in the german manpage translation.
    Thanks to Chris Leick and Georg Koppen! (Closes: #574962)
* apt-pkg/contrib/strutl.cc:
  - convert all toupper calls to tolower_ascii for a little speedup
* apt-pkg/contrib/strutl.cc:
  - always escape '%' (LP: #130289) (Closes: #500560)
  - unescape '%' sequence only if followed by 2 hex digit
  - username/password are urlencoded in proxy string (RFC 3986)

14 files changed, 1011 insertions, 346 deletions

diff --git a/apt-pkg/contrib/cmndline.cc b/apt-pkg/contrib/cmndline.cc
index bfd53695e..5a9944096 100644
--- a/apt-pkg/contrib/cmndline.cc
+++ b/apt-pkg/contrib/cmndline.cc
@@ -135,7 +135,9 @@ bool CommandLine::Parse(int argc,const char **argv)
    for (; I != argc; I++)
       *Files++ = argv[I];
    *Files = 0;
-   
+
+   SaveInConfig(argc, argv);
+
    return true;
 }
 									/*}}}*/
@@ -351,3 +353,41 @@ bool CommandLine::DispatchArg(Dispatch *Map,bool NoMatch)
    return false;
 }
 									/*}}}*/
+// CommandLine::SaveInConfig - for output later in a logfile or so	/*{{{*/
+// ---------------------------------------------------------------------
+/* We save the commandline here to have it around later for e.g. logging.
+   It feels a bit like a hack here and isn't bulletproof, but it is better
+   than nothing after all. */
+void CommandLine::SaveInConfig(unsigned int const &argc, char const * const * const argv)
+{
+   char cmdline[100 + argc * 50];
+   unsigned int length = 0;
+   bool lastWasOption = false;
+   bool closeQuote = false;
+   for (unsigned int i = 0; i < argc && length < sizeof(cmdline); ++i, ++length)
+   {
+      for (unsigned int j = 0; argv[i][j] != '\0' && length < sizeof(cmdline)-1; ++j, ++length)
+      {
+	 cmdline[length] = argv[i][j];
+	 if (lastWasOption == true && argv[i][j] == '=')
+	 {
+	    // That is possibly an option: Quote it if it includes spaces,
+	    // the benefit is that this will eliminate also most false positives
+	    const char* c = &argv[i][j+1];
+	    for (; *c != '\0' && *c != ' '; ++c);
+	    if (*c == '\0') continue;
+	    cmdline[++length] = '"';
+	    closeQuote = true;
+	 }
+      }
+      if (closeQuote == true)
+	 cmdline[length++] = '"';
+      // Problem: detects also --hello
+      if (cmdline[length-1] == 'o')
+	 lastWasOption = true;
+      cmdline[length] = ' ';
+   }
+   cmdline[--length] = '\0';
+   _config->Set("CommandLine::AsString", cmdline);
+}
+									/*}}}*/
diff --git a/apt-pkg/contrib/cmndline.h b/apt-pkg/contrib/cmndline.h
index e28071e81..7c0c71aa7 100644
--- a/apt-pkg/contrib/cmndline.h
+++ b/apt-pkg/contrib/cmndline.h
@@ -60,6 +60,7 @@ class CommandLine
    Configuration *Conf;
    bool HandleOpt(int &I,int argc,const char *argv[],
 		  const char *&Opt,Args *A,bool PreceedeMatch = false);
+   void static SaveInConfig(unsigned int const &argc, char const * const * const argv);
 
    public:
    
diff --git a/apt-pkg/contrib/configuration.cc b/apt-pkg/contrib/configuration.cc
index 7588b041c..81cc87d15 100644
--- a/apt-pkg/contrib/configuration.cc
+++ b/apt-pkg/contrib/configuration.cc
@@ -773,6 +773,8 @@ bool ReadConfigFile(Configuration &Conf,const string &FName,bool const &AsSectio
 	       else
 		  return _error->Error(_("Syntax error %s:%u: Unsupported directive '%s'"),FName.c_str(),CurLine,Tag.c_str());
 	    }
+	    else if (Tag.empty() == true && NoWord == false && Word == "#clear")
+	       return _error->Error(_("Syntax error %s:%u: clear directive requires an option tree as argument"),FName.c_str(),CurLine);
 	    else
 	    {
 	       // Set the item in the configuration class
@@ -841,3 +843,46 @@ bool ReadConfigDir(Configuration &Conf,const string &Dir,
    return true;
 }
 									/*}}}*/
+// MatchAgainstConfig Constructor					/*{{{*/
+Configuration::MatchAgainstConfig::MatchAgainstConfig(char const * Config)
+{
+   std::vector<std::string> const strings = _config->FindVector(Config);
+   for (std::vector<std::string>::const_iterator s = strings.begin();
+	s != strings.end(); ++s)
+   {
+      regex_t *p = new regex_t;
+      if (regcomp(p, s->c_str(), REG_EXTENDED | REG_ICASE | REG_NOSUB) == 0)
+	 patterns.push_back(p);
+      else
+      {
+	 regfree(p);
+	 delete p;
+	 _error->Warning("Regex compilation error for '%s' in configuration option '%s'",
+				s->c_str(), Config);
+      }
+    }
+
+}
+									/*}}}*/
+// MatchAgainstConfig Destructor					/*{{{*/
+Configuration::MatchAgainstConfig::~MatchAgainstConfig()
+{
+   for(std::vector<regex_t *>::const_iterator p = patterns.begin();
+	p != patterns.end(); ++p)
+   {
+      regfree(*p);
+      delete *p;
+   }
+}
+									/*}}}*/
+// MatchAgainstConfig::Match - returns true if a pattern matches	/*{{{*/
+bool Configuration::MatchAgainstConfig::Match(char const * str) const
+{
+   for(std::vector<regex_t *>::const_iterator p = patterns.begin();
+	p != patterns.end(); ++p)
+      if (regexec(*p, str, 0, 0, 0) == 0)
+	 return true;
+
+   return false;
+}
+									/*}}}*/
diff --git a/apt-pkg/contrib/configuration.h b/apt-pkg/contrib/configuration.h
index 2494c1d7c..cbe18e4e5 100644
--- a/apt-pkg/contrib/configuration.h
+++ b/apt-pkg/contrib/configuration.h
@@ -28,7 +28,7 @@
 #ifndef PKGLIB_CONFIGURATION_H
 #define PKGLIB_CONFIGURATION_H
 
-
+#include <regex.h>
 
 #include <string>
 #include <vector>
@@ -104,6 +104,23 @@ class Configuration
    Configuration(const Item *Root);
    Configuration();
    ~Configuration();
+
+   /** \brief match a string against a configurable list of patterns */
+   class MatchAgainstConfig
+   {
+     std::vector<regex_t *> patterns;
+
+   public:
+     MatchAgainstConfig(char const * Config);
+     virtual ~MatchAgainstConfig();
+
+     /** \brief Returns \b true for a string matching one of the patterns */
+     bool Match(char const * str) const;
+     bool Match(std::string const &str) const { return Match(str.c_str()); };
+
+     /** \brief returns if the matcher setup was successful */
+     bool wasConstructedSuccessfully() const { return patterns.empty() == false; }
+   };
 };
 
 extern Configuration *_config;
diff --git a/apt-pkg/contrib/error.cc b/apt-pkg/contrib/error.cc
index 927b7e05c..e2e8d6e57 100644
--- a/apt-pkg/contrib/error.cc
+++ b/apt-pkg/contrib/error.cc
@@ -1,16 +1,15 @@
 // -*- mode: cpp; mode: fold -*-
 // Description								/*{{{*/
-// $Id: error.cc,v 1.11 2002/03/26 07:38:58 jgg Exp $
 /* ######################################################################
-   
-   Global Erorr Class - Global error mechanism
+
+   Global Error Class - Global error mechanism
 
    We use a simple STL vector to store each error record. A PendingFlag
    is kept which indicates when the vector contains a Sever error.
-   
+
    This source is placed in the Public Domain, do with it what you will
    It was originally written by Jason Gunthorpe.
-   
+
    ##################################################################### */
 									/*}}}*/
 // Include Files							/*{{{*/
@@ -19,7 +18,6 @@
 #include <iostream>
 #include <errno.h>
 #include <stdio.h>
-#include <stdarg.h>
 #include <unistd.h>
 
 #include <string>
@@ -28,209 +26,233 @@
 #include "config.h"
    									/*}}}*/
 
-using namespace std;
-
 // Global Error Object							/*{{{*/
 /* If the implementation supports posix threads then the accessor function
    is compiled to be thread safe otherwise a non-safe version is used. A
    Per-Thread error object is maintained in much the same manner as libc
    manages errno */
 #if defined(_POSIX_THREADS) && defined(HAVE_PTHREAD)
- #include <pthread.h>
-
- static pthread_key_t ErrorKey;
- static void ErrorDestroy(void *Obj) {delete (GlobalError *)Obj;};
- static void KeyAlloc() {pthread_key_create(&ErrorKey,ErrorDestroy);};
-
- GlobalError *_GetErrorObj()
- {
-    static pthread_once_t Once = PTHREAD_ONCE_INIT;
-    pthread_once(&Once,KeyAlloc);
-    
-    void *Res = pthread_getspecific(ErrorKey);
-    if (Res == 0)
-       pthread_setspecific(ErrorKey,Res = new GlobalError);
-    return (GlobalError *)Res;
- }
+	#include <pthread.h>
+
+	static pthread_key_t ErrorKey;
+	static void ErrorDestroy(void *Obj) {delete (GlobalError *)Obj;};
+	static void KeyAlloc() {pthread_key_create(&ErrorKey,ErrorDestroy);};
+
+	GlobalError *_GetErrorObj() {
+		static pthread_once_t Once = PTHREAD_ONCE_INIT;
+		pthread_once(&Once,KeyAlloc);
+
+		void *Res = pthread_getspecific(ErrorKey);
+		if (Res == 0)
+			pthread_setspecific(ErrorKey,Res = new GlobalError);
+		return (GlobalError *)Res;
+	}
 #else
- GlobalError *_GetErrorObj()
- {
-    static GlobalError *Obj = new GlobalError;
-    return Obj;
- }
+	GlobalError *_GetErrorObj() {
+		static GlobalError *Obj = new GlobalError;
+		return Obj;
+	}
 #endif
 									/*}}}*/
-
 // GlobalError::GlobalError - Constructor				/*{{{*/
-// ---------------------------------------------------------------------
-/* */
-GlobalError::GlobalError() : List(0), PendingFlag(false)
-{
+GlobalError::GlobalError() : PendingFlag(false) {}
+									/*}}}*/
+// GlobalError::FatalE - Get part of the error string from errno	/*{{{*/
+bool GlobalError::FatalE(const char *Function,const char *Description,...) {
+	va_list args;
+	va_start(args,Description);
+	return InsertErrno(FATAL, Function, Description, args);
 }
 									/*}}}*/
 // GlobalError::Errno - Get part of the error string from errno		/*{{{*/
-// ---------------------------------------------------------------------
-/* Function indicates the stdlib function that failed and Description is
-   a user string that leads the text. Form is:
-     Description - Function (errno: strerror)
-   Carefull of the buffer overrun, sprintf.
- */
-bool GlobalError::Errno(const char *Function,const char *Description,...)
+bool GlobalError::Errno(const char *Function,const char *Description,...) {
+	va_list args;
+	va_start(args,Description);
+	return InsertErrno(ERROR, Function, Description, args);
+}
+									/*}}}*/
+// GlobalError::WarningE - Get part of the warning string from errno	/*{{{*/
+bool GlobalError::WarningE(const char *Function,const char *Description,...) {
+	va_list args;
+	va_start(args,Description);
+	return InsertErrno(WARNING, Function, Description, args);
+}
+									/*}}}*/
+// GlobalError::NoticeE - Get part of the notice string from errno	/*{{{*/
+bool GlobalError::NoticeE(const char *Function,const char *Description,...) {
+	va_list args;
+	va_start(args,Description);
+	return InsertErrno(NOTICE, Function, Description, args);
+}
+									/*}}}*/
+// GlobalError::DebugE - Get part of the debug string from errno	/*{{{*/
+bool GlobalError::DebugE(const char *Function,const char *Description,...) {
+	va_list args;
+	va_start(args,Description);
+	return InsertErrno(DEBUG, Function, Description, args);
+}
+									/*}}}*/
+// GlobalError::InsertErrno - Get part of the errortype string from errno/*{{{*/
+bool GlobalError::InsertErrno(MsgType const &type, const char *Function,
+				const char *Description,...) {
+	va_list args;
+	va_start(args,Description);
+	return InsertErrno(type, Function, Description, args);
+}
+									/*}}}*/
+// GlobalError::InsertErrno - formats an error message with the errno	/*{{{*/
+bool GlobalError::InsertErrno(MsgType type, const char* Function,
+			      const char* Description, va_list &args) {
+	char S[400];
+	snprintf(S, sizeof(S), "%s - %s (%i: %s)", Description,
+		 Function, errno, strerror(errno));
+	return Insert(type, S, args);
+}
+									/*}}}*/
+// GlobalError::Fatal - Add a fatal error to the list			/*{{{*/
+bool GlobalError::Fatal(const char *Description,...) {
+	va_list args;
+	va_start(args,Description);
+	return Insert(FATAL, Description, args);
+}
+									/*}}}*/
+// GlobalError::Error - Add an error to the list			/*{{{*/
+bool GlobalError::Error(const char *Description,...) {
+	va_list args;
+	va_start(args,Description);
+	return Insert(ERROR, Description, args);
+}
+									/*}}}*/
+// GlobalError::Warning - Add a warning to the list			/*{{{*/
+bool GlobalError::Warning(const char *Description,...) {
+	va_list args;
+	va_start(args,Description);
+	return Insert(WARNING, Description, args);
+}
+									/*}}}*/
+// GlobalError::Notice - Add a notice to the list			/*{{{*/
+bool GlobalError::Notice(const char *Description,...)
+{
+	va_list args;
+	va_start(args,Description);
+	return Insert(NOTICE, Description, args);
+}
+									/*}}}*/
+// GlobalError::Debug - Add a debug to the list				/*{{{*/
+bool GlobalError::Debug(const char *Description,...)
 {
-   va_list args;
-   va_start(args,Description);
-
-   // sprintf the description
-   char S[400];
-   vsnprintf(S,sizeof(S),Description,args);
-   snprintf(S + strlen(S),sizeof(S) - strlen(S),
-	    " - %s (%i: %s)",Function,errno,strerror(errno));
-
-   // Put it on the list
-   Item *Itm = new Item;
-   Itm->Text = S;
-   Itm->Error = true;
-   Insert(Itm);
-
-   PendingFlag = true;
-
-   return false;
-}
-									/*}}}*/
-// GlobalError::WarningE - Get part of the warn string from errno	/*{{{*/
-// ---------------------------------------------------------------------
-/* Function indicates the stdlib function that failed and Description is
-   a user string that leads the text. Form is:
-     Description - Function (errno: strerror)
-   Carefull of the buffer overrun, sprintf.
- */
-bool GlobalError::WarningE(const char *Function,const char *Description,...)
+	va_list args;
+	va_start(args,Description);
+	return Insert(DEBUG, Description, args);
+}
+									/*}}}*/
+// GlobalError::Insert - Add a errotype message to the list		/*{{{*/
+bool GlobalError::Insert(MsgType const &type, const char *Description,...)
 {
-   va_list args;
-   va_start(args,Description);
+	va_list args;
+	va_start(args,Description);
+	return Insert(type, Description, args);
+}
+									/*}}}*/
+// GlobalError::Insert - Insert a new item at the end			/*{{{*/
+bool GlobalError::Insert(MsgType type, const char* Description,
+			 va_list &args) {
+	char S[400];
+	vsnprintf(S,sizeof(S),Description,args);
+
+	Item const m(S, type);
+	Messages.push_back(m);
 
-   // sprintf the description
-   char S[400];
-   vsnprintf(S,sizeof(S),Description,args);
-   snprintf(S + strlen(S),sizeof(S) - strlen(S),
-	    " - %s (%i: %s)",Function,errno,strerror(errno));
+	if (type == ERROR || type == FATAL)
+		PendingFlag = true;
 
-   // Put it on the list
-   Item *Itm = new Item;
-   Itm->Text = S;
-   Itm->Error = false;
-   Insert(Itm);
+	if (type == FATAL || type == DEBUG)
+		std::clog << m << std::endl;
 
-   return false;
+	return false;
 }
 									/*}}}*/
-// GlobalError::Error - Add an error to the list			/*{{{*/
-// ---------------------------------------------------------------------
-/* Just vsprintfs and pushes */
-bool GlobalError::Error(const char *Description,...)
-{
-   va_list args;
-   va_start(args,Description);
+// GlobalError::PopMessage - Pulls a single message out			/*{{{*/
+bool GlobalError::PopMessage(std::string &Text) {
+	if (Messages.empty() == true)
+		return false;
+
+	Item const msg = Messages.front();
+	Messages.pop_front();
 
-   // sprintf the description
-   char S[400];
-   vsnprintf(S,sizeof(S),Description,args);
+	bool const Ret = (msg.Type == ERROR || msg.Type == FATAL);
+	Text = msg.Text;
+	if (PendingFlag == false || Ret == false)
+		return Ret;
 
-   // Put it on the list
-   Item *Itm = new Item;
-   Itm->Text = S;
-   Itm->Error = true;
-   Insert(Itm);
-   
-   PendingFlag = true;
-   
-   return false;
+	// check if another error message is pending
+	for (std::list<Item>::const_iterator m = Messages.begin();
+	     m != Messages.end(); m++)
+		if (m->Type == ERROR || m->Type == FATAL)
+			return Ret;
+
+	PendingFlag = false;
+	return Ret;
 }
 									/*}}}*/
-// GlobalError::Warning - Add a warning to the list			/*{{{*/
-// ---------------------------------------------------------------------
-/* This doesn't set the pending error flag */
-bool GlobalError::Warning(const char *Description,...)
-{
-   va_list args;
-   va_start(args,Description);
+// GlobalError::DumpErrors - Dump all of the errors/warns to cerr	/*{{{*/
+void GlobalError::DumpErrors(std::ostream &out, MsgType const &threshold,
+			     bool const &mergeStack) {
+	if (mergeStack == true)
+		for (std::list<MsgStack>::const_reverse_iterator s = Stacks.rbegin();
+		     s != Stacks.rend(); ++s)
+			Messages.insert(Messages.begin(), s->Messages.begin(), s->Messages.end());
 
-   // sprintf the description
-   char S[400];
-   vsnprintf(S,sizeof(S),Description,args);
+	for (std::list<Item>::const_iterator m = Messages.begin();
+	     m != Messages.end(); m++)
+		if (m->Type >= threshold)
+			out << (*m) << std::endl;
+	Discard();
+}
+									/*}}}*/
+// GlobalError::Discard - Discard					/*{{{*/
+void GlobalError::Discard() {
+	Messages.clear();
+	PendingFlag = false;
+};
+									/*}}}*/
+// GlobalError::empty - does our error list include anything?		/*{{{*/
+bool GlobalError::empty(MsgType const &trashhold) const {
+	if (PendingFlag == true)
+		return false;
 
-   // Put it on the list
-   Item *Itm = new Item;
-   Itm->Text = S;
-   Itm->Error = false;
-   Insert(Itm);
-   
-   return false;
+	if (Messages.empty() == true)
+		return true;
+
+	for (std::list<Item>::const_iterator m = Messages.begin();
+	     m != Messages.end(); m++)
+		if (m->Type >= trashhold)
+			return false;
+
+	return true;
 }
 									/*}}}*/
-// GlobalError::PopMessage - Pulls a single message out			/*{{{*/
-// ---------------------------------------------------------------------
-/* This should be used in a loop checking empty() each cycle. It returns
-   true if the message is an error. */
-bool GlobalError::PopMessage(string &Text)
-{
-   if (List == 0)
-      return false;
-      
-   bool Ret = List->Error;
-   Text = List->Text;
-   Item *Old = List;
-   List = List->Next;
-   delete Old;
-   
-   // This really should check the list to see if only warnings are left..
-   if (List == 0)
-      PendingFlag = false;
-   
-   return Ret;
+// GlobalError::PushToStack						/*{{{*/
+void GlobalError::PushToStack() {
+	MsgStack pack(Messages, PendingFlag);
+	Stacks.push_back(pack);
+	Discard();
 }
 									/*}}}*/
-// GlobalError::DumpErrors - Dump all of the errors/warns to cerr	/*{{{*/
-// ---------------------------------------------------------------------
-/* */
-void GlobalError::DumpErrors()
-{
-   // Print any errors or warnings found
-   string Err;
-   while (empty() == false)
-   {
-      bool Type = PopMessage(Err);
-      if (Type == true)
-	 cerr << "E: " << Err << endl;
-      else
-	 cerr << "W: " << Err << endl;
-   }
-}
-									/*}}}*/
-// GlobalError::Discard - Discard									/*{{{*/
-// ---------------------------------------------------------------------
-/* */
-void GlobalError::Discard()
-{
-   while (List != 0)
-   {
-      Item *Old = List;
-      List = List->Next;
-      delete Old;
-   }
-   
-   PendingFlag = false;
-};
+// GlobalError::RevertToStack						/*{{{*/
+void GlobalError::RevertToStack() {
+	Discard();
+	MsgStack pack = Stacks.back();
+	Messages = pack.Messages;
+	PendingFlag = pack.PendingFlag;
+	Stacks.pop_back();
+}
 									/*}}}*/
-// GlobalError::Insert - Insert a new item at the end			/*{{{*/
-// ---------------------------------------------------------------------
-/* */
-void GlobalError::Insert(Item *Itm)
-{
-   Item **End = &List;
-   for (Item *I = List; I != 0; I = I->Next)
-      End = &I->Next;
-   Itm->Next = *End;
-   *End = Itm;
+// GlobalError::MergeWithStack						/*{{{*/
+void GlobalError::MergeWithStack() {
+	MsgStack pack = Stacks.back();
+	Messages.insert(Messages.begin(), pack.Messages.begin(), pack.Messages.end());
+	PendingFlag = PendingFlag || pack.PendingFlag;
+	Stacks.pop_back();
 }
 									/*}}}*/
diff --git a/apt-pkg/contrib/error.h b/apt-pkg/contrib/error.h
index 90747ff7e..ae756dbc4 100644
--- a/apt-pkg/contrib/error.h
+++ b/apt-pkg/contrib/error.h
@@ -42,43 +42,277 @@
 
 #include <apt-pkg/macros.h>
 
+#include <iostream>
+#include <list>
 #include <string>
 
-class GlobalError
+#include <stdarg.h>
+
+class GlobalError							/*{{{*/
 {
-   struct Item
-   {
-      std::string Text;
-      bool Error;
-      Item *Next;
-   };
-   
-   Item *List;
-   bool PendingFlag;
-   void Insert(Item *I);
-   
-   public:
+public:									/*{{{*/
+	/** \brief a message can have one of following severity */
+	enum MsgType {
+		/** \brief Message will be printed instantly as it is likely that
+			this error will lead to a complete crash */
+		FATAL = 40,
+		/** \brief An error does hinder the correct execution and should be corrected */
+		ERROR = 30,
+		/** \brief indicates problem that can lead to errors later on */
+		WARNING = 20,
+		/** \brief deprecation warnings, old fallback behavior, … */
+		NOTICE = 10,
+		/** \brief for developers only in areas it is hard to print something directly */
+		DEBUG = 0
+	};
 
-   // Call to generate an error from a library call.
-   bool Errno(const char *Function,const char *Description,...) __like_printf_2 __cold;
-   bool WarningE(const char *Function,const char *Description,...) __like_printf_2 __cold;
+	/** \brief add a fatal error message with errno to the list
+	 *
+	 *  \param Function name of the function generating the error
+	 *  \param Description format string for the error message
+	 *
+	 *  \return \b false
+	 */
+	bool FatalE(const char *Function,const char *Description,...) __like_printf(3) __cold;
 
-   /* A warning should be considered less severe than an error, and may be
-      ignored by the client. */
-   bool Error(const char *Description,...) __like_printf_1 __cold;
-   bool Warning(const char *Description,...) __like_printf_1 __cold;
+	/** \brief add an Error message with errno to the list
+	 *
+	 *  \param Function name of the function generating the error
+	 *  \param Description format string for the error message
+	 *
+	 *  \return \b false
+	 */
+	bool Errno(const char *Function,const char *Description,...) __like_printf(3) __cold;
 
-   // Simple accessors
-   inline bool PendingError() {return PendingFlag;};
-   inline bool empty() {return List == 0;};
-   bool PopMessage(std::string &Text);
-   void Discard();
+	/** \brief add a warning message with errno to the list
+	 *
+	 *  A warning should be considered less severe than an error and
+	 *  may be ignored by the client.
+	 *
+	 *  \param Function Name of the function generates the warning.
+	 *  \param Description Format string for the warning message.
+	 *
+	 *  \return \b false
+	 */
+	bool WarningE(const char *Function,const char *Description,...) __like_printf(3) __cold;
 
-   // Usefull routine to dump to cerr
-   void DumpErrors();
-   
-   GlobalError();
+	/** \brief add a notice message with errno to the list
+	 *
+	 *  \param Function name of the function generating the error
+	 *  \param Description format string for the error message
+	 *
+	 *  \return \b false
+	 */
+	bool NoticeE(const char *Function,const char *Description,...) __like_printf(3) __cold;
+
+	/** \brief add a debug message with errno to the list
+	 *
+	 *  \param Function name of the function generating the error
+	 *  \param Description format string for the error message
+	 *
+	 *  \return \b false
+	 */
+	bool DebugE(const char *Function,const char *Description,...) __like_printf(3) __cold;
+
+	/** \brief adds an errno message with the given type
+	 *
+	 * \param type of the error message
+	 * \param Function which failed
+	 * \param Description of the error
+	 */
+	bool InsertErrno(MsgType const &type, const char* Function,
+			 const char* Description,...) __like_printf(4) __cold;
+
+	/** \brief add an fatal error message to the list
+	 *
+	 *  Most of the stuff we consider as "error" is also "fatal" for
+	 *  the user as the application will not have the expected result,
+	 *  but a fatal message here means that it gets printed directly
+	 *  to stderr in addiction to adding it to the list as the error
+	 *  leads sometimes to crashes and a maybe duplicated message
+	 *  is better than "Segfault" as the only displayed text
+	 *
+	 *  \param Description Format string for the fatal error message.
+	 *
+	 *  \return \b false
+	 */
+	bool Fatal(const char *Description,...) __like_printf(2) __cold;
+
+	/** \brief add an Error message to the list
+	 *
+	 *  \param Description Format string for the error message.
+	 *
+	 *  \return \b false
+	 */
+	bool Error(const char *Description,...) __like_printf(2) __cold;
+
+	/** \brief add a warning message to the list
+	 *
+	 *  A warning should be considered less severe than an error and
+	 *  may be ignored by the client.
+	 *
+	 *  \param Description Format string for the message
+	 *
+	 *  \return \b false
+	 */
+	bool Warning(const char *Description,...) __like_printf(2) __cold;
+
+	/** \brief add a notice message to the list
+	 *
+	 *  A notice should be considered less severe than an error or a
+	 *  warning and can be ignored by the client without further problems
+	 *  for some times, but he should consider fixing the problem.
+	 *  This error type can be used for e.g. deprecation warnings of options.
+	 *
+	 *  \param Description Format string for the message
+	 *
+	 *  \return \b false
+	 */
+	bool Notice(const char *Description,...) __like_printf(2) __cold;
+
+	/** \brief add a debug message to the list
+	 *
+	 *  \param Description Format string for the message
+	 *
+	 *  \return \b false
+	 */
+	bool Debug(const char *Description,...) __like_printf(2) __cold;
+
+	/** \brief adds an error message with the given type
+	 *
+	 * \param type of the error message
+	 * \param Description of the error
+	 */
+	bool Insert(MsgType const &type, const char* Description,...) __like_printf(3) __cold;
+
+	/** \brief is an error in the list?
+	 *
+	 *  \return \b true if an error is included in the list, \b false otherwise
+	 */
+	inline bool PendingError() const {return PendingFlag;};
+
+	/** \brief is the list empty?
+	 *
+	 *  The default checks if the list is empty or contains only notices,
+	 *  if you want to check if also no notices happend set the parameter
+	 *  flag to \b false.
+	 *
+	 *  \param WithoutNotice does notices count, default is \b true, so no
+	 *
+	 *  \return \b true if an the list is empty, \b false otherwise
+	 */
+	bool empty(MsgType const &trashhold = WARNING) const;
+
+	/** \brief returns and removes the first (or last) message in the list
+	 *
+	 *  \param[out] Text message of the first/last item
+	 *
+	 *  \return \b true if the message was an error, \b false otherwise
+	 */
+	bool PopMessage(std::string &Text);
+
+	/** \brief clears the list of messages */
+	void Discard();
+
+	/** \brief outputs the list of messages to the given stream
+	 *
+	 *  Note that all messages are discarded, also the notices
+	 *  displayed or not.
+	 *
+	 *  \param[out] out output stream to write the messages in
+	 *  \param threshold minimim level considered
+         *  \param mergeStack 
+	 */
+	void DumpErrors(std::ostream &out, MsgType const &threshold = WARNING,
+			bool const &mergeStack = true);
+
+	/** \brief dumps the list of messages to std::cerr
+	 *
+	 *  Note that all messages are discarded, also the notices
+	 *  displayed or not.
+	 *
+	 *  \param threshold minimum level printed
+	 */
+	void inline DumpErrors(MsgType const &threshold) {
+		DumpErrors(std::cerr, threshold);
+	}
+
+        // mvo: we do this instead of using a default parameter in the
+        //      previous declaration to avoid a (subtle) API break for
+        //      e.g. sigc++ and mem_fun0
+	/** \brief dumps the messages of type WARNING or higher to std::cerr
+	 *
+	 *  Note that all messages are discarded, displayed or not.
+	 *
+	 */
+	void inline DumpErrors() {
+                DumpErrors(WARNING);
+	}
+
+	/** \brief put the current Messages into the stack
+	 *
+	 *  All "old" messages will be pushed into a stack to
+	 *  them later back, but for now the Message query will be
+	 *  empty and performs as no messages were present before.
+	 *
+	 * The stack can be as deep as you want - all stack operations
+	 * will only operate on the last element in the stack.
+	 */
+	void PushToStack();
+
+	/** \brief throw away all current messages */
+	void RevertToStack();
+
+	/** \brief merge current and stack together */
+	void MergeWithStack();
+
+	/** \brief return the deep of the stack */
+	size_t StackCount() const {
+		return Stacks.size();
+	}
+
+	GlobalError();
+									/*}}}*/
+private:								/*{{{*/
+	struct Item {
+		std::string Text;
+		MsgType Type;
+
+		Item(char const *Text, MsgType const &Type) :
+			Text(Text), Type(Type) {};
+
+		friend std::ostream& operator<< (std::ostream &out, Item i) {
+			switch(i.Type) {
+			case FATAL:
+			case ERROR: out << "E"; break;
+			case WARNING: out << "W"; break;
+			case NOTICE: out << "N"; break;
+			case DEBUG: out << "D"; break;
+			}
+			return out << ": " << i.Text;
+		}
+	};
+
+	std::list<Item> Messages;
+	bool PendingFlag;
+
+	struct MsgStack {
+		std::list<Item> const Messages;
+		bool const PendingFlag;
+
+		MsgStack(std::list<Item> const &Messages, bool const &Pending) :
+			 Messages(Messages), PendingFlag(Pending) {};
+	};
+
+	std::list<MsgStack> Stacks;
+
+	bool InsertErrno(MsgType type, const char* Function,
+			 const char* Description, va_list &args);
+	bool Insert(MsgType type, const char* Description,
+			 va_list &args);
+									/*}}}*/
 };
+									/*}}}*/
 
 // The 'extra-ansi' syntax is used to help with collisions. 
 GlobalError *_GetErrorObj();
diff --git a/apt-pkg/contrib/fileutl.cc b/apt-pkg/contrib/fileutl.cc
index da32983f1..2a3b8a87d 100644
--- a/apt-pkg/contrib/fileutl.cc
+++ b/apt-pkg/contrib/fileutl.cc
@@ -11,6 +11,7 @@
    Most of this source is placed in the Public Domain, do with it what 
    you will
    It was originally written by Jason Gunthorpe <jgg@debian.org>.
+   FileFd gzip support added by Martin Pitt <martin.pitt@canonical.com>
    
    The exception is RunScripts() it is under the GPLv2
 
@@ -18,6 +19,7 @@
 									/*}}}*/
 // Include Files							/*{{{*/
 #include <apt-pkg/fileutl.h>
+#include <apt-pkg/strutl.h>
 #include <apt-pkg/error.h>
 #include <apt-pkg/sptr.h>
 #include <apt-pkg/configuration.h>
@@ -26,6 +28,7 @@
 
 #include <cstdlib>
 #include <cstring>
+#include <cstdio>
 
 #include <iostream>
 #include <unistd.h>
@@ -197,16 +200,62 @@ bool FileExists(string File)
    return true;
 }
 									/*}}}*/
+// DirectoryExists - Check if a directory exists and is really one	/*{{{*/
+// ---------------------------------------------------------------------
+/* */
+bool DirectoryExists(string const &Path)
+{
+   struct stat Buf;
+   if (stat(Path.c_str(),&Buf) != 0)
+      return false;
+   return ((Buf.st_mode & S_IFDIR) != 0);
+}
+									/*}}}*/
+// CreateDirectory - poor man's mkdir -p guarded by a parent directory	/*{{{*/
+// ---------------------------------------------------------------------
+/* This method will create all directories needed for path in good old
+   mkdir -p style but refuses to do this if Parent is not a prefix of
+   this Path. Example: /var/cache/ and /var/cache/apt/archives are given,
+   so it will create apt/archives if /var/cache exists - on the other
+   hand if the parent is /var/lib the creation will fail as this path
+   is not a parent of the path to be generated. */
+bool CreateDirectory(string const &Parent, string const &Path)
+{
+   if (Parent.empty() == true || Path.empty() == true)
+      return false;
+
+   if (DirectoryExists(Path) == true)
+      return true;
+
+   if (DirectoryExists(Parent) == false)
+      return false;
+
+   // we are not going to create directories "into the blue"
+   if (Path.find(Parent, 0) != 0)
+      return false;
+
+   vector<string> const dirs = VectorizeString(Path.substr(Parent.size()), '/');
+   string progress = Parent;
+   for (vector<string>::const_iterator d = dirs.begin(); d != dirs.end(); ++d)
+   {
+      if (d->empty() == true)
+	 continue;
+
+      progress.append("/").append(*d);
+      if (DirectoryExists(progress) == true)
+	 continue;
+
+      if (mkdir(progress.c_str(), 0755) != 0)
+	 return false;
+   }
+   return true;
+}
+									/*}}}*/
 // GetListOfFilesInDir - returns a vector of files in the given dir	/*{{{*/
 // ---------------------------------------------------------------------
 /* If an extension is given only files with this extension are included
    in the returned vector, otherwise every "normal" file is included. */
 std::vector<string> GetListOfFilesInDir(string const &Dir, string const &Ext,
-					bool const &SortList)
-{
-   return GetListOfFilesInDir(Dir, Ext, SortList, false);
-}
-std::vector<string> GetListOfFilesInDir(string const &Dir, string const &Ext,
 					bool const &SortList, bool const &AllowNoExt)
 {
    std::vector<string> ext;
@@ -234,6 +283,7 @@ std::vector<string> GetListOfFilesInDir(string const &Dir, std::vector<string> c
    }
 
    std::vector<string> List;
+   Configuration::MatchAgainstConfig SilentIgnore("Dir::Ignore-Files-Silently");
    DIR *D = opendir(Dir.c_str());
    if (D == 0) 
    {
@@ -259,6 +309,7 @@ std::vector<string> GetListOfFilesInDir(string const &Dir, std::vector<string> c
 	    {
 	       if (Debug == true)
 		  std::clog << "Bad file: " << Ent->d_name << " → no extension" << std::endl;
+	       _error->Notice("Ignoring file '%s' in directory '%s' as it has no filename extension", Ent->d_name, Dir.c_str());
 	       continue;
 	    }
 	 }
@@ -266,6 +317,8 @@ std::vector<string> GetListOfFilesInDir(string const &Dir, std::vector<string> c
 	 {
 	    if (Debug == true)
 	       std::clog << "Bad file: " << Ent->d_name << " → bad extension »" << flExtension(Ent->d_name) << "«" << std::endl;
+	    if (SilentIgnore.Match(Ent->d_name) == false)
+	       _error->Notice("Ignoring file '%s' in directory '%s' as it has an invalid filename extension", Ent->d_name, Dir.c_str());
 	    continue;
 	 }
       }
@@ -604,13 +657,25 @@ bool FileFd::Open(string FileName,OpenMode Mode, unsigned long Perms)
       case ReadOnly:
       iFd = open(FileName.c_str(),O_RDONLY);
       break;
+
+      case ReadOnlyGzip:
+      iFd = open(FileName.c_str(),O_RDONLY);
+      if (iFd > 0) {
+	 gz = gzdopen (iFd, "r");
+	 if (gz == NULL) {
+	     close (iFd);
+	     iFd = -1;
+	 }
+      }
+      break;
       
       case WriteEmpty:
       {
-	 struct stat Buf;
-	 if (lstat(FileName.c_str(),&Buf) == 0 && S_ISLNK(Buf.st_mode))
-	    unlink(FileName.c_str());
-	 iFd = open(FileName.c_str(),O_RDWR | O_CREAT | O_TRUNC,Perms);
+	 Flags |= Replace;
+	 char *name = strdup((FileName + ".XXXXXX").c_str());
+	 TemporaryFileName = string(mktemp(name));
+	 iFd = open(TemporaryFileName.c_str(),O_RDWR | O_CREAT | O_EXCL,Perms);
+	 free(name);
 	 break;
       }
       
@@ -635,6 +700,24 @@ bool FileFd::Open(string FileName,OpenMode Mode, unsigned long Perms)
    SetCloseExec(iFd,true);
    return true;
 }
+
+bool FileFd::OpenDescriptor(int Fd, OpenMode Mode, bool AutoClose)
+{
+   Close();
+   Flags = (AutoClose) ? FileFd::AutoClose : 0;
+   iFd = Fd;
+   if (Mode == ReadOnlyGzip) {
+      gz = gzdopen (iFd, "r");
+      if (gz == NULL) {
+	 if (AutoClose)
+	    close (iFd);
+	 return _error->Errno("gzdopen",_("Could not open file descriptor %d"),
+			      Fd);
+      }
+   }
+   this->FileName = "";
+   return true;
+}
 									/*}}}*/
 // FileFd::~File - Closes the file					/*{{{*/
 // ---------------------------------------------------------------------
@@ -658,7 +741,10 @@ bool FileFd::Read(void *To,unsigned long Size,unsigned long *Actual)
    
    do
    {
-      Res = read(iFd,To,Size);
+      if (gz != NULL)
+         Res = gzread(gz,To,Size);
+      else
+         Res = read(iFd,To,Size);
       if (Res < 0 && errno == EINTR)
 	 continue;
       if (Res < 0)
@@ -697,7 +783,10 @@ bool FileFd::Write(const void *From,unsigned long Size)
    errno = 0;
    do
    {
-      Res = write(iFd,From,Size);
+      if (gz != NULL)
+         Res = gzwrite(gz,From,Size);
+      else
+         Res = write(iFd,From,Size);
       if (Res < 0 && errno == EINTR)
 	 continue;
       if (Res < 0)
@@ -723,7 +812,12 @@ bool FileFd::Write(const void *From,unsigned long Size)
 /* */
 bool FileFd::Seek(unsigned long To)
 {
-   if (lseek(iFd,To,SEEK_SET) != (signed)To)
+   int res;
+   if (gz)
+      res = gzseek(gz,To,SEEK_SET);
+   else
+      res = lseek(iFd,To,SEEK_SET);
+   if (res != (signed)To)
    {
       Flags |= Fail;
       return _error->Error("Unable to seek to %lu",To);
@@ -737,7 +831,12 @@ bool FileFd::Seek(unsigned long To)
 /* */
 bool FileFd::Skip(unsigned long Over)
 {
-   if (lseek(iFd,Over,SEEK_CUR) < 0)
+   int res;
+   if (gz)
+      res = gzseek(gz,Over,SEEK_CUR);
+   else
+      res = lseek(iFd,Over,SEEK_CUR);
+   if (res < 0)
    {
       Flags |= Fail;
       return _error->Error("Unable to seek ahead %lu",Over);
@@ -751,6 +850,11 @@ bool FileFd::Skip(unsigned long Over)
 /* */
 bool FileFd::Truncate(unsigned long To)
 {
+   if (gz)
+   {
+      Flags |= Fail;
+      return _error->Error("Truncating gzipped files is not implemented (%s)", FileName.c_str());
+   }
    if (ftruncate(iFd,To) != 0)
    {
       Flags |= Fail;
@@ -765,7 +869,11 @@ bool FileFd::Truncate(unsigned long To)
 /* */
 unsigned long FileFd::Tell()
 {
-   off_t Res = lseek(iFd,0,SEEK_CUR);
+   off_t Res;
+   if (gz)
+     Res = gztell(gz);
+   else
+     Res = lseek(iFd,0,SEEK_CUR);
    if (Res == (off_t)-1)
       _error->Errno("lseek","Failed to determine the current file position");
    return Res;
@@ -776,6 +884,7 @@ unsigned long FileFd::Tell()
 /* */
 unsigned long FileFd::Size()
 {
+   //TODO: For gz, do we need the actual file size here or the uncompressed length?
    struct stat Buf;
    if (fstat(iFd,&Buf) != 0)
       return _error->Errno("fstat","Unable to determine the file size");
@@ -789,14 +898,33 @@ bool FileFd::Close()
 {
    bool Res = true;
    if ((Flags & AutoClose) == AutoClose)
-      if (iFd >= 0 && close(iFd) != 0)
-	 Res &= _error->Errno("close",_("Problem closing the file"));
+   {
+      if (gz != NULL) {
+	 int const e = gzclose(gz);
+	 // gzdopen() on empty files always fails with "buffer error" here, ignore that
+	 if (e != 0 && e != Z_BUF_ERROR)
+	    Res &= _error->Errno("close",_("Problem closing the gzip file %s"), FileName.c_str());
+      } else
+	 if (iFd > 0 && close(iFd) != 0)
+	    Res &= _error->Errno("close",_("Problem closing the file %s"), FileName.c_str());
+   }
+
+   if ((Flags & Replace) == Replace && iFd >= 0) {
+      if (rename(TemporaryFileName.c_str(), FileName.c_str()) != 0)
+	 Res &= _error->Errno("rename",_("Problem renaming the file %s to %s"), TemporaryFileName.c_str(), FileName.c_str());
+
+      FileName = TemporaryFileName; // for the unlink() below.
+   }
+
    iFd = -1;
-   
+   gz = NULL;
+
    if ((Flags & Fail) == Fail && (Flags & DelOnFail) == DelOnFail &&
        FileName.empty() == false)
       if (unlink(FileName.c_str()) != 0)
-	 Res &= _error->WarningE("unlnk",_("Problem unlinking the file"));
+	 Res &= _error->WarningE("unlnk",_("Problem unlinking the file %s"), FileName.c_str());
+
+
    return Res;
 }
 									/*}}}*/
diff --git a/apt-pkg/contrib/fileutl.h b/apt-pkg/contrib/fileutl.h
index 85a94898c..62705478d 100644
--- a/apt-pkg/contrib/fileutl.h
+++ b/apt-pkg/contrib/fileutl.h
@@ -21,10 +21,16 @@
 #ifndef PKGLIB_FILEUTL_H
 #define PKGLIB_FILEUTL_H
 
+#include <apt-pkg/macros.h>
 
 #include <string>
 #include <vector>
 
+#include <zlib.h>
+
+/* Define this for python-apt */
+#define APT_HAS_GZIP 1
+
 using std::string;
 
 class FileFd
@@ -33,12 +39,14 @@ class FileFd
    int iFd;
  
    enum LocalFlags {AutoClose = (1<<0),Fail = (1<<1),DelOnFail = (1<<2),
-                    HitEof = (1<<3)};
+                    HitEof = (1<<3), Replace = (1<<4) };
    unsigned long Flags;
    string FileName;
-   
+   string TemporaryFileName;
+   gzFile gz;
+
    public:
-   enum OpenMode {ReadOnly,WriteEmpty,WriteExists,WriteAny,WriteTemp};
+   enum OpenMode {ReadOnly,WriteEmpty,WriteExists,WriteAny,WriteTemp,ReadOnlyGzip};
    
    inline bool Read(void *To,unsigned long Size,bool AllowEof)
    {
@@ -55,6 +63,7 @@ class FileFd
    unsigned long Tell();
    unsigned long Size();
    bool Open(string FileName,OpenMode Mode,unsigned long Perms = 0666);
+   bool OpenDescriptor(int Fd, OpenMode Mode, bool AutoClose=false);
    bool Close();
    bool Sync();
    
@@ -69,12 +78,12 @@ class FileFd
    inline string &Name() {return FileName;};
    
    FileFd(string FileName,OpenMode Mode,unsigned long Perms = 0666) : iFd(-1), 
-            Flags(0) 
+            Flags(0), gz(NULL)
    {
       Open(FileName,Mode,Perms);
    };
-   FileFd(int Fd = -1) : iFd(Fd), Flags(AutoClose) {};
-   FileFd(int Fd,bool) : iFd(Fd), Flags(0) {};
+   FileFd(int Fd = -1) : iFd(Fd), Flags(AutoClose), gz(NULL) {};
+   FileFd(int Fd,bool) : iFd(Fd), Flags(0), gz(NULL) {};
    virtual ~FileFd();
 };
 
@@ -82,11 +91,10 @@ bool RunScripts(const char *Cnf);
 bool CopyFile(FileFd &From,FileFd &To);
 int GetLock(string File,bool Errors = true);
 bool FileExists(string File);
-// FIXME: next ABI-Break: merge the two method-headers
-std::vector<string> GetListOfFilesInDir(string const &Dir, string const &Ext,
-					bool const &SortList);
+bool DirectoryExists(string const &Path) __attrib_const;
+bool CreateDirectory(string const &Parent, string const &Path);
 std::vector<string> GetListOfFilesInDir(string const &Dir, string const &Ext,
-					bool const &SortList, bool const &AllowNoExt);
+					bool const &SortList, bool const &AllowNoExt=false);
 std::vector<string> GetListOfFilesInDir(string const &Dir, std::vector<string> const &Ext,
 					bool const &SortList);
 string SafeGetCWD();
diff --git a/apt-pkg/contrib/macros.h b/apt-pkg/contrib/macros.h
index 9aeb77b81..62e7b65db 100644
--- a/apt-pkg/contrib/macros.h
+++ b/apt-pkg/contrib/macros.h
@@ -56,33 +56,35 @@
 
 // some nice optional GNUC features
 #if __GNUC__ >= 3
-        #define __must_check    __attribute__ ((warn_unused_result))
-        #define __deprecated    __attribute__ ((deprecated))
-        /* likely() and unlikely() can be used to mark boolean expressions
-           as (not) likely true which will help the compiler to optimise */
-        #define likely(x)       __builtin_expect (!!(x), 1)
-        #define unlikely(x)     __builtin_expect (!!(x), 0)
+	#define __must_check	__attribute__ ((warn_unused_result))
+	#define __deprecated	__attribute__ ((deprecated))
+	#define __attrib_const	__attribute__ ((__const__))
+	/* likely() and unlikely() can be used to mark boolean expressions
+	   as (not) likely true which will help the compiler to optimise */
+	#define likely(x)	__builtin_expect (!!(x), 1)
+	#define unlikely(x)	__builtin_expect (!!(x), 0)
 #else
-        #define __must_check    /* no warn_unused_result */
-        #define __deprecated    /* no deprecated */
-        #define likely(x)       (x)
-        #define unlikely(x)     (x)
+	#define __must_check	/* no warn_unused_result */
+	#define __deprecated	/* no deprecated */
+	#define __attrib_const	/* no const attribute */
+	#define likely(x)	(x)
+	#define unlikely(x)	(x)
 #endif
 
 // cold functions are unlikely() to be called
 #if (__GNUC__ == 4 && __GNUC_MINOR__ >= 3) || __GNUC__ > 4
-        #define __cold  __attribute__ ((__cold__))
+	#define __cold	__attribute__ ((__cold__))
+	#define __hot	__attribute__ ((__hot__))
 #else
-        #define __cold  /* no cold marker */
+	#define __cold	/* no cold marker */
+	#define __hot	/* no hot marker */
 #endif
 
 #ifdef __GNUG__
 // Methods have a hidden this parameter that is visible to this attribute
-	#define __like_printf_1 __attribute__ ((format (printf, 2, 3)))
-	#define __like_printf_2 __attribute__ ((format (printf, 3, 4)))
+	#define __like_printf(n)	__attribute__((format(printf, n, n + 1)))
 #else
-	#define __like_printf_1
-	#define __like_printf_2
+	#define __like_printf(n)	/* no like-printf */
 #endif
 
 #endif
diff --git a/apt-pkg/contrib/mmap.cc b/apt-pkg/contrib/mmap.cc
index f440f9489..69fb61fca 100644
--- a/apt-pkg/contrib/mmap.cc
+++ b/apt-pkg/contrib/mmap.cc
@@ -27,6 +27,7 @@
 #include <unistd.h>
 #include <fcntl.h>
 #include <stdlib.h>
+#include <errno.h>
 
 #include <cstring>
    									/*}}}*/
@@ -35,7 +36,7 @@
 // ---------------------------------------------------------------------
 /* */
 MMap::MMap(FileFd &F,unsigned long Flags) : Flags(Flags), iSize(0),
-                     Base(0)
+                     Base(0), SyncToFd(NULL)
 {
    if ((Flags & NoImmMap) != NoImmMap)
       Map(F);
@@ -45,7 +46,7 @@ MMap::MMap(FileFd &F,unsigned long Flags) : Flags(Flags), iSize(0),
 // ---------------------------------------------------------------------
 /* */
 MMap::MMap(unsigned long Flags) : Flags(Flags), iSize(0),
-                     Base(0)
+                     Base(0), SyncToFd(NULL)
 {
 }
 									/*}}}*/
@@ -78,7 +79,24 @@ bool MMap::Map(FileFd &Fd)
    // Map it.
    Base = mmap(0,iSize,Prot,Map,Fd.Fd(),0);
    if (Base == (void *)-1)
-      return _error->Errno("mmap",_("Couldn't make mmap of %lu bytes"),iSize);
+   {
+      if (errno == ENODEV || errno == EINVAL)
+      {
+	 // The filesystem doesn't support this particular kind of mmap.
+	 // So we allocate a buffer and read the whole file into it.
+	 int const dupped_fd = dup(Fd.Fd());
+	 if (dupped_fd == -1)
+	    return _error->Errno("mmap", _("Couldn't duplicate file descriptor %i"), Fd.Fd());
+
+	 Base = new unsigned char[iSize];
+	 SyncToFd = new FileFd (dupped_fd);
+	 if (!SyncToFd->Seek(0L) || !SyncToFd->Read(Base, iSize))
+	    return false;
+      }
+      else
+	 return _error->Errno("mmap",_("Couldn't make mmap of %lu bytes"),
+	                      iSize);
+     }
 
    return true;
 }
@@ -93,10 +111,19 @@ bool MMap::Close(bool DoSync)
    
    if (DoSync == true)
       Sync();
-   
-   if (munmap((char *)Base,iSize) != 0)
-      _error->Warning("Unable to munmap");
-   
+
+   if (SyncToFd != NULL)
+   {
+      delete[] (char *)Base;
+      delete SyncToFd;
+      SyncToFd = NULL;
+   }
+   else
+   {
+      if (munmap((char *)Base, iSize) != 0)
+	 _error->WarningE("mmap", _("Unable to close mmap"));
+   }
+
    iSize = 0;
    Base = 0;
    return true;
@@ -113,8 +140,18 @@ bool MMap::Sync()
    
 #ifdef _POSIX_SYNCHRONIZED_IO   
    if ((Flags & ReadOnly) != ReadOnly)
-      if (msync((char *)Base,iSize,MS_SYNC) < 0)
-	 return _error->Errno("msync","Unable to write mmap");
+   {
+      if (SyncToFd != NULL)
+      {
+	 if (!SyncToFd->Seek(0) || !SyncToFd->Write(Base, iSize))
+	    return false;
+      }
+      else
+      {
+	 if (msync((char *)Base, iSize, MS_SYNC) < 0)
+	    return _error->Errno("msync", _("Unable to synchronize mmap"));
+      }
+   }
 #endif   
    return true;
 }
@@ -130,8 +167,19 @@ bool MMap::Sync(unsigned long Start,unsigned long Stop)
 #ifdef _POSIX_SYNCHRONIZED_IO
    unsigned long PSize = sysconf(_SC_PAGESIZE);
    if ((Flags & ReadOnly) != ReadOnly)
-      if (msync((char *)Base+(int)(Start/PSize)*PSize,Stop - Start,MS_SYNC) < 0)
-	 return _error->Errno("msync","Unable to write mmap");
+   {
+      if (SyncToFd != 0)
+      {
+	 if (!SyncToFd->Seek(0) ||
+	     !SyncToFd->Write (((char *)Base)+Start, Stop-Start))
+	    return false;
+      }
+      else
+      {
+	 if (msync((char *)Base+(int)(Start/PSize)*PSize,Stop - Start,MS_SYNC) < 0)
+	    return _error->Errno("msync", _("Unable to synchronize mmap"));
+      }
+   }
 #endif   
    return true;
 }
@@ -177,22 +225,22 @@ DynamicMMap::DynamicMMap(unsigned long Flags,unsigned long const &WorkSpace,
 
 	// disable Moveable if we don't grow
 	if (Grow == 0)
-		Flags &= ~Moveable;
+		this->Flags &= ~Moveable;
 
 #ifndef __linux__
 	// kfreebsd doesn't have mremap, so we use the fallback
-	if ((Flags & Moveable) == Moveable)
-		Flags |= Fallback;
+	if ((this->Flags & Moveable) == Moveable)
+		this->Flags |= Fallback;
 #endif
 
 #ifdef _POSIX_MAPPED_FILES
-	if ((Flags & Fallback) != Fallback) {
+	if ((this->Flags & Fallback) != Fallback) {
 		// Set the permissions.
 		int Prot = PROT_READ;
 		int Map = MAP_PRIVATE | MAP_ANONYMOUS;
-		if ((Flags & ReadOnly) != ReadOnly)
+		if ((this->Flags & ReadOnly) != ReadOnly)
 			Prot |= PROT_WRITE;
-		if ((Flags & Public) == Public)
+		if ((this->Flags & Public) == Public)
 			Map = MAP_SHARED | MAP_ANONYMOUS;
 
 		// use anonymous mmap() to get the memory
@@ -249,7 +297,7 @@ unsigned long DynamicMMap::RawAllocate(unsigned long Size,unsigned long Aln)
    {
       if(!Grow())
       {
-	 _error->Error(_("Dynamic MMap ran out of room. Please increase the size "
+	 _error->Fatal(_("Dynamic MMap ran out of room. Please increase the size "
 			 "of APT::Cache-Limit. Current value: %lu. (man 5 apt.conf)"), WorkSpace);
 	 return 0;
       }
@@ -266,7 +314,7 @@ unsigned long DynamicMMap::Allocate(unsigned long ItemSize)
    // Look for a matching pool entry
    Pool *I;
    Pool *Empty = 0;
-   for (I = Pools; I != Pools + PoolCount; I++)
+   for (I = Pools; I != Pools + PoolCount; ++I)
    {
       if (I->ItemSize == 0)
 	 Empty = I;
@@ -294,7 +342,11 @@ unsigned long DynamicMMap::Allocate(unsigned long ItemSize)
    {
       const unsigned long size = 20*1024;
       I->Count = size/ItemSize;
+      Pool* oldPools = Pools;
       Result = RawAllocate(size,ItemSize);
+      if (Pools != oldPools)
+	 I += Pools - oldPools;
+
       // Does the allocation failed ?
       if (Result == 0 && _error->PendingError())
 	 return 0;
@@ -317,7 +369,7 @@ unsigned long DynamicMMap::WriteString(const char *String,
    if (Len == (unsigned long)-1)
       Len = strlen(String);
 
-   unsigned long Result = RawAllocate(Len+1,0);
+   unsigned long const Result = RawAllocate(Len+1,0);
 
    if (Result == 0 && _error->PendingError())
       return 0;
@@ -344,19 +396,25 @@ unsigned long DynamicMMap::WriteString(const char *String,
    the nearly impossible 4 to grow it before it finally give up: Never say never. */
 bool DynamicMMap::Grow() {
 	if (Limit != 0 && WorkSpace >= Limit)
-		return _error->Error(_("The size of a MMap has already reached the defined limit of %lu bytes,"
-		                       "abort the try to grow the MMap."), Limit);
+		return _error->Error(_("Unable to increase the size of the MMap as the "
+		                       "limit of %lu bytes is already reached."), Limit);
+	if (GrowFactor <= 0)
+		return _error->Error(_("Unable to increase size of the MMap as automatic growing is disabled by user."));
 
-	unsigned long const newSize = WorkSpace + 1024*1024;
+	unsigned long const newSize = WorkSpace + GrowFactor;
 
 	if(Fd != 0) {
 		Fd->Seek(newSize - 1);
 		char C = 0;
 		Fd->Write(&C,sizeof(C));
 	}
+
+	unsigned long const poolOffset = Pools - ((Pool*) Base);
+
 	if ((Flags & Fallback) != Fallback) {
 #if defined(_POSIX_MAPPED_FILES) && defined(__linux__)
    #ifdef MREMAP_MAYMOVE
+
 		if ((Flags & Moveable) == Moveable)
 			Base = mremap(Base, WorkSpace, newSize, MREMAP_MAYMOVE);
 		else
@@ -377,6 +435,7 @@ bool DynamicMMap::Grow() {
 			return false;
 	}
 
+	Pools =(Pool*) Base + poolOffset;
 	WorkSpace = newSize;
 	return true;
 }
diff --git a/apt-pkg/contrib/mmap.h b/apt-pkg/contrib/mmap.h
index cd2b15ba2..5ca951204 100644
--- a/apt-pkg/contrib/mmap.h
+++ b/apt-pkg/contrib/mmap.h
@@ -44,6 +44,11 @@ class MMap
    unsigned long iSize;
    void *Base;
 
+   // In case mmap can not be used, we keep a dup of the file
+   // descriptor that should have been mmaped so that we can write to
+   // the file in Sync().
+   FileFd *SyncToFd;
+
    bool Map(FileFd &Fd);
    bool Close(bool DoSync = true);
    
diff --git a/apt-pkg/contrib/strutl.cc b/apt-pkg/contrib/strutl.cc
index 1b9922a31..ace74cb37 100644
--- a/apt-pkg/contrib/strutl.cc
+++ b/apt-pkg/contrib/strutl.cc
@@ -198,7 +198,8 @@ bool ParseQuoteWord(const char *&String,string &Res)
    char *I;
    for (I = Buffer; I < Buffer + sizeof(Buffer) && Start != C; I++)
    {
-      if (*Start == '%' && Start + 2 < C)
+      if (*Start == '%' && Start + 2 < C &&
+	  isxdigit(Start[1]) && isxdigit(Start[2]))
       {
 	 Tmp[0] = Start[1];
 	 Tmp[1] = Start[2];
@@ -273,7 +274,8 @@ string QuoteString(const string &Str, const char *Bad)
    for (string::const_iterator I = Str.begin(); I != Str.end(); I++)
    {
       if (strchr(Bad,*I) != 0 || isprint(*I) == 0 || 
-	  *I <= 0x20 || *I >= 0x7F)
+	  *I == 0x25 || // percent '%' char
+	  *I <= 0x20 || *I >= 0x7F) // control chars
       {
 	 char Buf[10];
 	 sprintf(Buf,"%%%02x",(int)*I);
@@ -290,10 +292,16 @@ string QuoteString(const string &Str, const char *Bad)
 /* This undoes QuoteString */
 string DeQuoteString(const string &Str)
 {
+   return DeQuoteString(Str.begin(),Str.end());
+}
+string DeQuoteString(string::const_iterator const &begin,
+			string::const_iterator const &end)
+{
    string Res;
-   for (string::const_iterator I = Str.begin(); I != Str.end(); I++)
+   for (string::const_iterator I = begin; I != end; I++)
    {
-      if (*I == '%' && I + 2 < Str.end())
+      if (*I == '%' && I + 2 < end &&
+	  isxdigit(I[1]) && isxdigit(I[2]))
       {
 	 char Tmp[3];
 	 Tmp[0] = I[1];
@@ -566,7 +574,7 @@ int stringcmp(string::const_iterator A,string::const_iterator AEnd,
 int stringcasecmp(const char *A,const char *AEnd,const char *B,const char *BEnd)
 {
    for (; A != AEnd && B != BEnd; A++, B++)
-      if (toupper(*A) != toupper(*B))
+      if (tolower_ascii(*A) != tolower_ascii(*B))
 	 break;
 
    if (A == AEnd && B == BEnd)
@@ -575,7 +583,7 @@ int stringcasecmp(const char *A,const char *AEnd,const char *B,const char *BEnd)
       return 1;
    if (B == BEnd)
       return -1;
-   if (toupper(*A) < toupper(*B))
+   if (tolower_ascii(*A) < tolower_ascii(*B))
       return -1;
    return 1;
 }
@@ -584,7 +592,7 @@ int stringcasecmp(string::const_iterator A,string::const_iterator AEnd,
 		  const char *B,const char *BEnd)
 {
    for (; A != AEnd && B != BEnd; A++, B++)
-      if (toupper(*A) != toupper(*B))
+      if (tolower_ascii(*A) != tolower_ascii(*B))
 	 break;
 
    if (A == AEnd && B == BEnd)
@@ -593,7 +601,7 @@ int stringcasecmp(string::const_iterator A,string::const_iterator AEnd,
       return 1;
    if (B == BEnd)
       return -1;
-   if (toupper(*A) < toupper(*B))
+   if (tolower_ascii(*A) < tolower_ascii(*B))
       return -1;
    return 1;
 }
@@ -601,7 +609,7 @@ int stringcasecmp(string::const_iterator A,string::const_iterator AEnd,
 		  string::const_iterator B,string::const_iterator BEnd)
 {
    for (; A != AEnd && B != BEnd; A++, B++)
-      if (toupper(*A) != toupper(*B))
+      if (tolower_ascii(*A) != tolower_ascii(*B))
 	 break;
 
    if (A == AEnd && B == BEnd)
@@ -610,7 +618,7 @@ int stringcasecmp(string::const_iterator A,string::const_iterator AEnd,
       return 1;
    if (B == BEnd)
       return -1;
-   if (toupper(*A) < toupper(*B))
+   if (tolower_ascii(*A) < tolower_ascii(*B))
       return -1;
    return 1;
 }
@@ -789,28 +797,28 @@ bool ReadMessages(int Fd, vector<string> &List)
 // MonthConv - Converts a month string into a number			/*{{{*/
 // ---------------------------------------------------------------------
 /* This was lifted from the boa webserver which lifted it from 'wn-v1.07'
-   Made it a bit more robust with a few touppers though. */
+   Made it a bit more robust with a few tolower_ascii though. */
 static int MonthConv(char *Month)
 {
-   switch (toupper(*Month)) 
+   switch (tolower_ascii(*Month)) 
    {
-      case 'A':
-      return toupper(Month[1]) == 'P'?3:7;
-      case 'D':
+      case 'a':
+      return tolower_ascii(Month[1]) == 'p'?3:7;
+      case 'd':
       return 11;
-      case 'F':
+      case 'f':
       return 1;
-      case 'J':
-      if (toupper(Month[1]) == 'A')
+      case 'j':
+      if (tolower_ascii(Month[1]) == 'a')
 	 return 0;
-      return toupper(Month[2]) == 'N'?5:6;
-      case 'M':
-      return toupper(Month[2]) == 'R'?2:4;
-      case 'N':
+      return tolower_ascii(Month[2]) == 'n'?5:6;
+      case 'm':
+      return tolower_ascii(Month[2]) == 'r'?2:4;
+      case 'n':
       return 10;
-      case 'O':
+      case 'o':
       return 9;
-      case 'S':
+      case 's':
       return 8;
 
       // Pretend it is January..
@@ -819,34 +827,70 @@ static int MonthConv(char *Month)
    }   
 }
 									/*}}}*/
-// timegm - Internal timegm function if gnu is not available		/*{{{*/
+// timegm - Internal timegm if the gnu version is not available		/*{{{*/
 // ---------------------------------------------------------------------
-/* Ripped this evil little function from wget - I prefer the use of 
-   GNU timegm if possible as this technique will have interesting problems
-   with leap seconds, timezones and other.
-   
-   Converts struct tm to time_t, assuming the data in tm is UTC rather
+/* Converts struct tm to time_t, assuming the data in tm is UTC rather
    than local timezone (mktime assumes the latter).
-   
-   Contributed by Roger Beeman <beeman@cisco.com>, with the help of
-   Mark Baushke <mdb@cisco.com> and the rest of the Gurus at CISCO. */
-
-/* Turned it into an autoconf check, because GNU is not the only thing which
-   can provide timegm. -- 2002-09-22, Joel Baker */
 
-#ifndef HAVE_TIMEGM // Now with autoconf!
+   This function is a nonstandard GNU extension that is also present on
+   the BSDs and maybe other systems. For others we follow the advice of
+   the manpage of timegm and use his portable replacement. */
+#ifndef HAVE_TIMEGM
 static time_t timegm(struct tm *t)
 {
-   time_t tl, tb;
-   
-   tl = mktime (t);
-   if (tl == -1)
-      return -1;
-   tb = mktime (gmtime (&tl));
-   return (tl <= tb ? (tl + (tl - tb)) : (tl - (tb - tl)));
+   char *tz = getenv("TZ");
+   setenv("TZ", "", 1);
+   tzset();
+   time_t ret = mktime(t);
+   if (tz)
+      setenv("TZ", tz, 1);
+   else
+      unsetenv("TZ");
+   tzset();
+   return ret;
 }
 #endif
 									/*}}}*/
+// FullDateToTime - Converts a HTTP1.1 full date strings into a time_t	/*{{{*/
+// ---------------------------------------------------------------------
+/* tries to parses a full date as specified in RFC2616 Section 3.3.1
+   with one exception: All timezones (%Z) are accepted but the protocol
+   says that it MUST be GMT, but this one is equal to UTC which we will
+   encounter from time to time (e.g. in Release files) so we accept all
+   here and just assume it is GMT (or UTC) later on */
+bool RFC1123StrToTime(const char* const str,time_t &time)
+{
+   struct tm Tm;
+   setlocale (LC_ALL,"C");
+   bool const invalid =
+   // Sun, 06 Nov 1994 08:49:37 GMT  ; RFC 822, updated by RFC 1123
+      (strptime(str, "%a, %d %b %Y %H:%M:%S %Z", &Tm) == NULL &&
+   // Sunday, 06-Nov-94 08:49:37 GMT ; RFC 850, obsoleted by RFC 1036
+       strptime(str, "%A, %d-%b-%y %H:%M:%S %Z", &Tm) == NULL &&
+   // Sun Nov  6 08:49:37 1994       ; ANSI C's asctime() format
+       strptime(str, "%a %b %d %H:%M:%S %Y", &Tm) == NULL);
+   setlocale (LC_ALL,"");
+   if (invalid == true)
+      return false;
+
+   time = timegm(&Tm);
+   return true;
+}
+									/*}}}*/
+// FTPMDTMStrToTime - Converts a ftp modification date into a time_t	/*{{{*/
+// ---------------------------------------------------------------------
+/* */
+bool FTPMDTMStrToTime(const char* const str,time_t &time)
+{
+   struct tm Tm;
+   // MDTM includes no whitespaces but recommend and ignored by strptime
+   if (strptime(str, "%Y %m %d %H %M %S", &Tm) == NULL)
+      return false;
+
+   time = timegm(&Tm);
+   return true;
+}
+									/*}}}*/
 // StrToTime - Converts a string into a time_t				/*{{{*/
 // ---------------------------------------------------------------------
 /* This handles all 3 populare time formats including RFC 1123, RFC 1036
@@ -1000,12 +1044,12 @@ bool TokSplitString(char Tok,char *Input,char **List,
    return true;
 }
 									/*}}}*/
-// ExplodeString - Split a string up into a vector			/*{{{*/
+// VectorizeString - Split a string up into a vector of strings		/*{{{*/
 // ---------------------------------------------------------------------
 /* This can be used to split a given string up into a vector, so the
    propose is the same as in the method above and this one is a bit slower
-   also, but the advantage is that we an iteratable vector */
-vector<string> ExplodeString(string const &haystack, char const &split)
+   also, but the advantage is that we have an iteratable vector */
+vector<string> VectorizeString(string const &haystack, char const &split)
 {
    string::const_iterator start = haystack.begin();
    string::const_iterator end = start;
@@ -1133,10 +1177,13 @@ char *safe_snprintf(char *Buffer,char *End,const char *Format,...)
 
 // tolower_ascii - tolower() function that ignores the locale		/*{{{*/
 // ---------------------------------------------------------------------
-/* */
-int tolower_ascii(int c)
+/* This little function is the most called method we have and tries
+   therefore to do the absolut minimum - and is noteable faster than
+   standard tolower/toupper and as a bonus avoids problems with different
+   locales - we only operate on ascii chars anyway. */
+int tolower_ascii(int const c)
 {
-   if (c >= 'A' and c <= 'Z')
+   if (c >= 'A' && c <= 'Z')
       return c + 32;
    return c;
 }
@@ -1235,9 +1282,10 @@ void URI::CopyFrom(const string &U)
    else
    {
       Host.assign(At+1,SingleSlash);
-      User.assign(FirstColon,SecondColon);
+      // username and password must be encoded (RFC 3986)
+      User.assign(DeQuoteString(FirstColon,SecondColon));
       if (SecondColon < At)
-	 Password.assign(SecondColon+1,At);
+	 Password.assign(DeQuoteString(SecondColon+1,At));
    }   
    
    // Now we parse the RFC 2732 [] hostnames.
diff --git a/apt-pkg/contrib/strutl.h b/apt-pkg/contrib/strutl.h
index e72288f4c..a457ff047 100644
--- a/apt-pkg/contrib/strutl.h
+++ b/apt-pkg/contrib/strutl.h
@@ -25,19 +25,12 @@
 #include <iostream>
 #include <time.h>
 
+#include "macros.h"
+
 using std::string;
 using std::vector;
 using std::ostream;
 
-#ifdef __GNUG__
-// Methods have a hidden this parameter that is visible to this attribute
-#define APT_FORMAT2 __attribute__ ((format (printf, 2, 3)))
-#define APT_FORMAT3 __attribute__ ((format (printf, 3, 4)))
-#else
-#define APT_FORMAT2
-#define APT_FORMAT3
-#endif    
-
 bool UTF8ToCodeset(const char *codeset, const string &orig, string *dest);
 char *_strstrip(char *String);
 char *_strtabexpand(char *String,size_t Len);
@@ -45,13 +38,16 @@ bool ParseQuoteWord(const char *&String,string &Res);
 bool ParseCWord(const char *&String,string &Res);
 string QuoteString(const string &Str,const char *Bad);
 string DeQuoteString(const string &Str);
+string DeQuoteString(string::const_iterator const &begin, string::const_iterator const &end);
 string SizeToStr(double Bytes);
 string TimeToStr(unsigned long Sec);
 string Base64Encode(const string &Str);
 string OutputInDepth(const unsigned long Depth, const char* Separator="  ");
 string URItoFileName(const string &URI);
 string TimeRFC1123(time_t Date);
-bool StrToTime(const string &Val,time_t &Result);
+bool RFC1123StrToTime(const char* const str,time_t &time) __must_check;
+bool FTPMDTMStrToTime(const char* const str,time_t &time) __must_check;
+__deprecated bool StrToTime(const string &Val,time_t &Result);
 string LookupTag(const string &Message,const char *Tag,const char *Default = 0);
 int StringToBool(const string &Text,int Default = -1);
 bool ReadMessages(int Fd, vector<string> &List);
@@ -59,12 +55,12 @@ bool StrToNum(const char *Str,unsigned long &Res,unsigned Len,unsigned Base = 0)
 bool Hex2Num(const string &Str,unsigned char *Num,unsigned int Length);
 bool TokSplitString(char Tok,char *Input,char **List,
 		    unsigned long ListMax);
-vector<string> ExplodeString(string const &haystack, char const &split);
-void ioprintf(ostream &out,const char *format,...) APT_FORMAT2;
-void strprintf(string &out,const char *format,...) APT_FORMAT2;
-char *safe_snprintf(char *Buffer,char *End,const char *Format,...) APT_FORMAT3;
+vector<string> VectorizeString(string const &haystack, char const &split) __attrib_const;
+void ioprintf(ostream &out,const char *format,...) __like_printf(2);
+void strprintf(string &out,const char *format,...) __like_printf(2);
+char *safe_snprintf(char *Buffer,char *End,const char *Format,...) __like_printf(3);
 bool CheckDomainList(const string &Host, const string &List);
-int tolower_ascii(int c);
+int tolower_ascii(int const c) __attrib_const __hot;
 
 #define APT_MKSTRCMP(name,func) \
 inline int name(const char *A,const char *B) {return func(A,A+strlen(A),B,B+strlen(B));}; \
@@ -144,6 +140,4 @@ struct RxChoiceList
 unsigned long RegexChoice(RxChoiceList *Rxs,const char **ListBegin,
 		      const char **ListEnd);
 
-#undef APT_FORMAT2
-
 #endif
diff --git a/apt-pkg/contrib/weakptr.h b/apt-pkg/contrib/weakptr.h
new file mode 100644
index 000000000..5158e393c
--- /dev/null
+++ b/apt-pkg/contrib/weakptr.h
@@ -0,0 +1,62 @@
+/* weakptr.h - An object which supports weak pointers.
+ *
+ * Copyright (C) 2010 Julian Andres Klode <jak@debian.org>
+ * 
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
+ * MA 02110-1301, USA.
+ */
+
+#ifndef WEAK_POINTER_H
+#define WEAK_POINTER_H
+
+#include <set>
+/**
+ * Class for objects providing support for weak pointers.
+ *
+ * This class allows for the registration of certain pointers as weak,
+ * which will cause them to be set to NULL when the destructor of the
+ * object is called.
+ */
+class WeakPointable {
+private:
+    std::set<WeakPointable**> pointers;
+
+public:
+
+    /**
+     * Add a new weak pointer.
+     */
+    inline void AddWeakPointer(WeakPointable** weakptr) {
+       pointers.insert(weakptr);
+    }
+
+    /**
+     * Remove the weak pointer from the list of weak pointers.
+     */
+    inline void RemoveWeakPointer(WeakPointable **weakptr) {
+       pointers.erase(weakptr);
+    }
+
+    /**
+     * Deconstruct the object, set all weak pointers to NULL.
+     */
+    ~WeakPointable() {
+        std::set<WeakPointable**>::iterator iter = pointers.begin();
+        while (iter != pointers.end())
+            **(iter++) = NULL;
+    }
+};
+
+#endif // WEAK_POINTER_H