Merge remote-tracking branch 'upstream/debian/experimental' into feature/srv-records

19 files changed, 720 insertions, 197 deletions

diff --git a/doc/acquire-additional-files.txt b/doc/acquire-additional-files.txt
new file mode 100644
index 000000000..71ce7b0cb
--- /dev/null
+++ b/doc/acquire-additional-files.txt
@@ -0,0 +1,233 @@
+# Acquire additional files in 'update' operations
+
+The download and verification of data from multiple sources in different
+compression formats, with partial downloads and patches is an involved
+process which is hard to implement correctly and securely.
+
+APT frontends share the code and binaries to make this happen in libapt
+with the Acquire system, supported by helpers shipped in the apt package
+itself and additional transports in individual packages like
+apt-transport-https.
+
+For its own operation libapt needs or can make use of Packages, Sources
+and Translation-* files, which it will acquire by default, but
+a repository might contain more data files (e.g.  Contents) a frontend
+might want to use and would therefore need to be downloaded as well
+(e.g. apt-file).
+
+This file describes the configuration scheme such a frontend can use to
+instruct the Acquire system to download those additional files.
+
+# The Configuration Stanza
+
+The Acquire system uses the same configuration settings to implement the
+files it downloads by default. These settings are the default, but if
+they would be written in a configuration file the configuration
+instructing the Acquire system to download the Packages files would look
+like this (see also apt.conf(5) manpage for configuration file syntax):
+
+	Acquire::IndexTargets::deb::Packages {
+		MetaKey "$(COMPONENT)/binary-$(ARCHITECTURE)/Packages";
+		ShortDescription "Packages";
+		Description "$(SITE) $(RELEASE)/$(COMPONENT) $(ARCHITECTURE) Packages";
+
+		flatMetaKey "Packages";
+		flatDescription "$(SITE) $(RELEASE) Packages";
+
+		Optional "false";
+	};
+
+All files which should be downloaded (nicknamed 'Targets') are mentioned
+below the Acquire::IndexTargets scope. 'deb' is here the type of the
+sources.list entry the file should be acquired for. The only other
+supported value is hence 'deb-src'. Beware: You can't specify multiple
+types here and you can't download the same (evaluated) MetaKey from
+multiple types!
+
+After the type you can pick any valid and unique string which preferable
+refers to the file it downloads (In the example we picked 'Packages').
+This string is used as identifier for the target class and accessible as
+'Created-By' e.g. in the "apt-get indextargets" output as detailed below.
+
+All targets have three main properties you can define:
+* MetaKey: The identifier of the file to be downloaded as used in the
+  Release file.  It is also the relative location of the file from the
+  Release file.  You can neither download from a different server
+  entirely (absolute URI) nor access directories above the Release file
+  (e.g. "../../").
+* ShortDescription: Very short string intended to be displayed to the
+  user e.g.  while reporting progress. apt will e.g. use this string in
+  the last line to indicate progress of e.g. the download of a specific
+  item.
+* Description: A preferable human understandable and readable identifier
+  of which file is acquired exactly. Mainly used for progress reporting
+  and error messages. apt will e.g. use this string in the Get/Hit/Err
+  progress lines.
+
+Additional optional properties:
+* flat{MetaKey,Description}: APT supports two types of repositories:
+  dists-style repositories which are the default and by far the most
+  common which are named after the fact that the files are in an
+  elaborated directory structure.  In contrast a flat-style repositories
+  lumps all files together in one directory.  Support for these flat
+  repositories exists mainly for legacy purposes only.  It is therefore
+  recommend to not set these values.
+* Optional: The default value is 'true' and should be kept at this
+  value.  If enabled the acquire system will skip the download if the
+  file isn't mentioned in the Release file. Otherwise this is treated as
+  a hard error and the update process fails. Note that failures while
+  downloading (e.g. 404 or hash verification errors) are failures,
+  regardless of this setting.
+
+
+The acquire system will automatically choose to download a compressed
+file if it is available and uncompress it for you, just as it will also
+use pdiff patching if provided by the repository and enabled by the
+user. You only have to ensure that the Release file contains the
+information about the compressed files/pdiffs to make this happen.
+NO properties have to be set to enable this.
+
+# More examples
+
+The stanzas for Translation-* files as well as for Sources files would
+look like this:
+
+Acquire::IndexTargets {
+	deb::Translations {
+		MetaKey "$(COMPONENT)/i18n/Translation-$(LANGUAGE)";
+		ShortDescription "Translation-$(LANGUAGE)";
+		Description "$(SITE) $(RELEASE)/$(COMPONENT) Translation-$(LANGUAGE)";
+
+		flatMetaKey "$(LANGUAGE)";
+		flatDescription "$(SITE) $(RELEASE) Translation-$(LANGUAGE)";
+	};
+
+	deb-src::Sources {
+		MetaKey "$(COMPONENT)/source/Sources";
+		ShortDescription "Sources";
+		Description "$(SITE) $(RELEASE)/$(COMPONENT) Sources";
+
+		flatMetaKey "Sources";
+		flatDescription "$(SITE) $(RELEASE) Sources";
+
+		Optional "false";
+	};
+};
+
+# Substitution variables
+
+As seen in the examples, properties can contain placeholders filled in
+by the acquire system. The following variables are known; note that
+unknown variables have no default value nor are they touched: They are
+printed as-is.
+
+* $(SITE): An identifier of the site we access as seen in sources.list,
+  e.g. "http://example.org/debian" or "file:/path/to/a/repository". You
+  can't use this field in {,flat}MetaKey, it is for description proposes
+  only.
+* $(RELEASE): This is usually an archive- or codename, e.g. "stable" or
+  "stretch".  Note that flat-style repositories do not have a archive-
+  or codename per-se, so the value might very well be just "/" or so.
+  Again, as seen in the sources.list.
+* $(COMPONENT): as given in the sources.list, e.g. "main", "non-free" or
+  "universe".  Note that flat-style repositories again do not really
+  have a meaningful value here.
+* $(LANGUAGE): Values are all entries (expect "none") of configuration
+  option Acquire::Languages, e.g. "en", "de" or "de_AT".
+* $(ARCHITECTURE): Values are all entries of configuration option
+  APT::Architectures (potentially modified by sources.list options),
+  e.g. "amd64", "i386" or "armel" for the 'deb' type. In type 'deb-src'
+  this variable has the value "source".
+
+Note that while more variables might exist in the implementation, these
+are to be considered undefined and their usage strongly discouraged. If
+you have a need for other variables contact us.
+
+# Accessing files
+
+Do NOT hardcode specific file locations, names or compression types in
+your application! You will notice that the configuration options give
+you no choice over where the downloaded files will be stored. This is by
+design so multiple applications can download and use the same file
+rather than each and every one of them potentially downloads and uses
+its own copy somewhere on disk.
+
+"apt-get indextargets" can be used to get the location as well as other
+information about all files downloaded (aka: you will see Packages,
+Sources and Translation-* files here as well). Provide a line of the
+default output format as parameter to filter out all entries which do
+not have such a line. With --format, you can further more define your
+own output style. The variables are what you see in the output, just all
+uppercase and wrapped in $(), as in the configuration file.
+
+To get all the filenames of all Translation-en files you can e.g. call:
+	apt-get indextargets --format '$(FILENAME)' "Created-By: Translations" "Language: en"
+
+The line-based filtering and the formating is rather crude and feature-
+less by design, so it is recommend to use dedicated and more powerful
+tools like 'grep-dctrl'.
+
+Accessing this information via libapt is done by reading the
+sources.lists (pkgSourceList), iterating over the metaIndex objects this
+creates and calling GetIndexTargets() on them. See the sourcecode of
+"apt-get indextargets" for a complete example.
+
+Note that by default targets are not listed if they weren't downloaded.
+If you want to see all targets, you can use the --no-release-info, which
+also removes the Codename, Suite, Version, Origin, Label and Trusted
+fields from the output as these also display data which needs to be
+downloaded first and could hence be inaccurate [on the pro-side: This
+mode is faster as it doesn't require a valid binary cache to operate].
+The most notable difference perhaps is in the Filename field through: By
+default it indicates an existing file, potentially compressed (Hint:
+libapt users can use FileFd to open compressed files transparently). In
+the --no-release-info mode the indicated file doesn't need to exist and
+it will always refer to an uncompressed file, even if the index would be
+(or is) stored compressed.
+
+Remarks on fields only available in (default) --release-info mode:
+* Trusted: Denotes with a 'yes' or 'no' if the data in this file is
+  authenticated by a trustchain rooted in a trusted gpg key. You should
+  be careful with untrusted data and warn the user if you use it.
+* Codename, Suite, Version, Origin and Label are fields from the Release
+  file, are only present if they are present in the Release file and
+  contain the same data.
+
+Remarks on other available fields:
+* MetaKey, ShortDesc, Description, Site, Release: as defined
+  by the configuration and described further above.
+* Created-By: configuration entity responsible for this target
+* Target-Of: type of the sources.list entry
+* URI, Repo-URI: avoid using. Contains potentially username/password.
+  Prefer 'Site', especially for display.
+* Optional: Decodes the option of the same name from the configuration.
+  Note that it is using 'yes' and 'no' instead of 'true' and 'false'.
+* Language, Architecture, Component: as defined further above, but with
+  the catch that they might be missing if they don't effect the target
+  (aka: They weren't used while evaluating the MetaKey template).
+
+Again, additional fields might be visible in certain implementations,
+but you should avoid using them and instead talk to us about a portable
+implementation.
+
+# Multiple application requiring the same files
+
+It is highly encouraged that applications talk to each other and to us
+about which files they require. It is usually best to have a common
+package ship the configuration needed to get the files, but specific
+needs might require specific solutions. Again: talk to us.
+
+# Acquiring files not mentioned in the Release file
+
+You can't. This is by design as these files couldn't be verified to not
+be modified in transit, corrupted by the download process or simple if
+they are present at all on the server, which would require apt to probe
+for them. APT did this in the past for legacy reasons, we do not intend
+to go back to these dark times.
+
+This is also why you can't request files from a different server. It
+would have the additional problem that this server might not even be
+accessible (e.g. proxy settings) or that local sources (file:/, cdrom:/)
+start requesting online files…
+
+In other words: We would be opening Pandora's box.
diff --git a/doc/apt-cache.8.xml b/doc/apt-cache.8.xml
index a9f6c8da2..a8f1b4586 100644
--- a/doc/apt-cache.8.xml
+++ b/doc/apt-cache.8.xml
@@ -122,8 +122,7 @@ Reverse Provides:
        </listitem>
 
        <listitem><para><literal>Total distinct</literal> versions is the number of package versions 
-       found in the cache; this value is therefore at least equal to the 
-       number of total package names. If more than one distribution is being accessed
+       found in the cache. If more than one distribution is being accessed
        (for instance, "stable" and "unstable"), this value
        can be considerably larger than the number of total package names.</para>
        </listitem>
@@ -172,7 +171,7 @@ Reverse Provides:
      If <option>--full</option> is given
      then output identical to <literal>show</literal> is produced for each matched
      package, and if <option>--names-only</option> is given then the long description
-     is not searched, only the package name is.</para>
+     is not searched, only the package name and provided packages are.</para>
      <para>
      Separate arguments can be used to specify multiple search patterns that 
      are and'ed together.</para></listitem>
@@ -281,12 +280,23 @@ Reverse Provides:
 		   <term><option>--no-breaks</option></term>
 		   <term><option>--no-replaces</option></term>
 		   <term><option>--no-enhances</option></term>
-		   <listitem><para>Per default the <literal>depends</literal> and
-     <literal>rdepends</literal> print all dependencies. This can be tweaked with
+		   <listitem><para>Per default the <command>depends</command> and
+     <command>rdepends</command> print all dependencies. This can be tweaked with
      these flags which will omit the specified dependency type.
      Configuration Item: <literal>APT::Cache::Show<replaceable>DependencyType</replaceable></literal>
      e.g. <literal>APT::Cache::ShowRecommends</literal>.</para></listitem>
      </varlistentry>
+
+     <varlistentry><term><option>--implicit</option></term>
+	<listitem><para>Per default <command>depends</command> and <command>rdepends</command>
+	      print only dependencies explicitly expressed in the metadata. With this flag
+	      it will also show dependencies implicitely added based on the encountered data.
+	      A <literal>Conflicts: foo</literal> e.g. expresses implicitely that this package
+	      also conflicts with the package foo from any other architecture.
+	      Configuration Item: <literal>APT::Cache::ShowImplicit</literal>.
+	</para></listitem>
+     </varlistentry>
+
      <varlistentry><term><option>-f</option></term><term><option>--full</option></term>
      <listitem><para>Print full package records when searching. 
      Configuration Item: <literal>APT::Cache::ShowFull</literal>.</para></listitem>
@@ -308,7 +318,7 @@ Reverse Provides:
      </varlistentry>
 
      <varlistentry><term><option>--names-only</option></term><term><option>-n</option></term>
-     <listitem><para>Only search on the package names, not the long descriptions. 
+     <listitem><para>Only search on the package and provided package names, not the long descriptions.
      Configuration Item: <literal>APT::Cache::NamesOnly</literal>.</para></listitem>
      </varlistentry>
 
diff --git a/doc/apt-get.8.xml b/doc/apt-get.8.xml
index a372a0d30..785b4e9a8 100644
--- a/doc/apt-get.8.xml
+++ b/doc/apt-get.8.xml
@@ -214,7 +214,7 @@
      <filename>&cachedir;/archives/partial/</filename>.</para></listitem>
      </varlistentry>
 
-     <varlistentry><term><option>autoclean</option></term>
+     <varlistentry><term><option>autoclean</option> (and the <option>auto-clean</option> alias since 1.1)</term>
      <listitem><para>Like <literal>clean</literal>, <literal>autoclean</literal> clears out the local 
      repository of retrieved package files. The difference is that it only 
      removes package files that can no longer be downloaded, and are largely 
@@ -224,25 +224,36 @@
      erased if it is set to off.</para></listitem>
      </varlistentry>
 
-     <varlistentry><term><option>autoremove</option></term>
+     <varlistentry><term><option>autoremove</option> (and the <option>auto-remove</option> alias since 1.1)</term>
      <listitem><para><literal>autoremove</literal> is used to remove packages that were automatically
      installed to satisfy dependencies for other packages and are now no longer needed.</para></listitem>
      </varlistentry>
 
      <varlistentry><term><option>changelog</option></term>
-       <listitem><para><literal>changelog</literal> downloads a package changelog and displays
-           it through <command>sensible-pager</command>. The server name and base
-           directory is defined in the <literal>APT::Changelogs::Server</literal>
-	   variable (e.g. <ulink url="http://packages.debian.org/changelogs">packages.debian.org/changelogs</ulink> for
-	   Debian or <ulink url="http://changelogs.ubuntu.com/changelogs">changelogs.ubuntu.com/changelogs</ulink> for
-           Ubuntu).
-           By default it displays the changelog for the version that is
-           installed.  However, you can specify the same options as for
-           the <option>install</option> command.
-         </para>
+	<listitem><para><literal>changelog</literal> tries to download the
+	      changelog of a package and displays it through
+	      <command>sensible-pager</command>.  By default it
+	      displays the changelog for the version that is installed.
+	      However, you can specify the same options as for the
+	      <option>install</option> command.</para>
        </listitem>
      </varlistentry>
 
+     <varlistentry><term><option>indextargets</option></term>
+	<listitem><para>Displays by default a deb822 formatted listing of
+	      information about all data files (aka index targets) <command>apt-get
+		 update</command> would download. Supports a
+	      <option>--format</option> option to modify the output format as
+	      well as accepts lines of the default output to filter the records
+	      by. The command is mainly used as an interface for external tools
+	      working with APT to get information as well as filenames for
+	      downloaded files so they can use them as well instead of
+	      downloading them again on their own. Detailed documentation is
+	      omitted here and can instead be found in the source tree in
+	      <literal><filename>doc/acquire-additional-files.txt</filename></literal>.
+	   </para>
+	</listitem>
+     </varlistentry>
 
 
    </variablelist>
@@ -316,17 +327,15 @@
 		   <term><option>--dry-run</option></term>
 		   <term><option>--recon</option></term>
 		   <term><option>--no-act</option></term>
-     <listitem><para>No action; perform a simulation of events that would occur but do not
-     actually change the system. 
-     Configuration Item: <literal>APT::Get::Simulate</literal>.</para>
-
-     <para>Simulated runs performed as a user will automatically deactivate locking
-     (<literal>Debug::NoLocking</literal>), and if the option
-     <literal>APT::Get::Show-User-Simulation-Note</literal> is set
-     (as it is by default) a notice will also be displayed indicating that
-     this is only a simulation. Runs performed as root do not trigger either
-     NoLocking or the notice - superusers should know what they are doing
-     without further warnings from <literal>apt-get</literal>.</para>
+     <listitem><para>No action; perform a simulation of events that would occur
+	   based on the current system state but do not actually change the
+	   system. Locking will be disabled (<option>Debug::NoLocking</option>)
+	   so the system state could change while <command>apt-get</command> is
+	   running. Simulations can also be executed by non-root users which might
+	   not have read access to all apt configuration distorting the simulation.
+	   A notice expressing this warning is also shown by default for non-root
+	   users (<option>APT::Get::Show-User-Simulation-Note</option>).
+	   Configuration Item: <literal>APT::Get::Simulate</literal>.</para>
 
      <para>Simulated runs print out a series of lines, each representing a <command>dpkg</command>
      operation: configure (<literal>Conf</literal>), remove (<literal>Remv</literal>)
@@ -420,12 +429,36 @@
      Configuration Item: <literal>APT::Get::Only-Upgrade</literal>.</para></listitem>
      </varlistentry>
 
+     <varlistentry><term><option>--allow-downgrades</option></term>
+     <listitem><para>This is a dangerous option that will cause apt to continue
+     without prompting if it is doing downgrades. It
+     should not be used except in very special situations. Using
+     it can potentially destroy your system!
+     Configuration Item: <literal>APT::Get::allow-downgrades</literal>. Introduced in APT 1.1.</para></listitem>
+     </varlistentry>
+
+     <varlistentry><term><option>--allow-remove-essential</option></term>
+     <listitem><para>Force yes; this is a dangerous option that will cause apt to continue
+     without prompting if it is removing essentials. It
+     should not be used except in very special situations. Using
+     it can potentially destroy your system!
+     Configuration Item: <literal>APT::Get::allow-remove-essential</literal>. Introduced in APT 1.1.</para></listitem>
+     </varlistentry>
+
+     <varlistentry><term><option>--allow-change-held-packages</option></term>
+     <listitem><para>Force yes; this is a dangerous option that will cause apt to continue
+     without prompting if it is changing held packages. It
+     should not be used except in very special situations. Using
+     it can potentially destroy your system!
+     Configuration Item: <literal>APT::Get::allow-change-held-packages</literal>. Introduced in APT 1.1.</para></listitem>
+     </varlistentry>
+
      <varlistentry><term><option>--force-yes</option></term>
      <listitem><para>Force yes; this is a dangerous option that will cause apt to continue 
      without prompting if it is doing something potentially harmful. It 
      should not be used except in very special situations. Using 
      <literal>force-yes</literal> can potentially destroy your system! 
-     Configuration Item: <literal>APT::Get::force-yes</literal>.</para></listitem>
+     Configuration Item: <literal>APT::Get::force-yes</literal>. This is deprecated and replaced by <option>--allow-downgrades</option>, <option>--allow-remove-essential</option>, <option>--allow-change-held-packages</option> in 1.1. </para></listitem>
      </varlistentry>
 
      <varlistentry><term><option>--print-uris</option></term>
@@ -520,9 +553,13 @@
      </varlistentry>
 
      <varlistentry><term><option>--allow-unauthenticated</option></term>
-     <listitem><para>Ignore if packages can't be authenticated and don't prompt about it.
-     This is useful for tools like pbuilder.
-     Configuration Item: <literal>APT::Get::AllowUnauthenticated</literal>.</para></listitem>
+     <listitem><para>Ignore if packages can't be authenticated and don't prompt
+	   about it.  This can be useful while working with local repositories,
+	   but is a huge security risk if data authenticity isn't ensured in
+	   another way by the user itself.  The usage of the
+	   <option>Trusted</option> option for &sources-list; entries should
+	   usually be preferred over this global override.  Configuration Item:
+	   <literal>APT::Get::AllowUnauthenticated</literal>.</para></listitem>
      </varlistentry>
 
      <varlistentry><term><option>--no-allow-insecure-repositories</option></term>
diff --git a/doc/apt.8.xml b/doc/apt.8.xml
index 29bf96751..18b97f547 100644
--- a/doc/apt.8.xml
+++ b/doc/apt.8.xml
@@ -44,7 +44,8 @@
      display a list of packages. It supports shell pattern for matching 
      package names and the following options:
        <option>--installed</option>, 
-       <option>--upgradable</option>, 
+       <option>--upgradable</option>,
+       <option>--upgradeable</option>,
        <option>--all-versions</option>
      are supported.
      </para></listitem>
@@ -85,6 +86,11 @@
      installed instead of removed.</para></listitem>
      </varlistentry>
 
+     <varlistentry><term><option>autoremove</option> (and the <option>auto-remove</option> alias since 1.1)</term>
+     <listitem><para><literal>autoremove</literal> is used to remove packages that were automatically
+     installed to satisfy dependencies for other packages and are now no longer needed.</para></listitem>
+     </varlistentry>
+
      <varlistentry><term><option>edit-sources</option></term>
      <listitem><para><literal>edit-sources</literal> lets you edit
      your sources.list file and provides basic sanity checks.
diff --git a/doc/apt.conf.5.xml b/doc/apt.conf.5.xml
index efe986ea8..d5e185757 100644
--- a/doc/apt.conf.5.xml
+++ b/doc/apt.conf.5.xml
@@ -301,6 +301,8 @@ DPkg::Pre-Install-Pkgs {"/usr/sbin/dpkg-preconfigure --apt";};
      <literal>Valid-Until</literal> header, but if they don't or a
      stricter value is desired the <literal>Max-ValidTime</literal>
      option below can be used.
+     The <option>Check-Valid-Until</option> option of &sources-list; entries should be
+     preferred to disable the check selectively instead of using this global override.
      </para></listitem>
      </varlistentry>
 
@@ -312,7 +314,8 @@ DPkg::Pre-Install-Pkgs {"/usr/sbin/dpkg-preconfigure --apt";};
 	 the earlier date of the two is used as the expiration date.
 	 The default value is <literal>0</literal> which stands for "valid forever".
 	 Archive specific settings can be made by appending the label of the archive
-	 to the option name.
+	 to the option name. Preferably, the same can be achieved for specific
+	 &sources-list; entries by using the <option>Valid-Until-Max</option> option there.
 	 </para></listitem>
      </varlistentry>
 
@@ -324,7 +327,8 @@ DPkg::Pre-Install-Pkgs {"/usr/sbin/dpkg-preconfigure --apt";};
 	 frequently updated archive with a <literal>Valid-Until</literal> header
 	 instead of completely disabling the expiration date checking.
 	 Archive specific settings can and should be used by appending the label of
-	 the archive to the option name.
+	 the archive to the option name. Preferably, the same can be achieved for specific
+	 &sources-list; entries by using the <option>Valid-Until-Min</option> option there.
 	 </para></listitem>
      </varlistentry>
 
@@ -618,6 +622,33 @@ DPkg::Pre-Install-Pkgs {"/usr/sbin/dpkg-preconfigure --apt";};
          </para></listitem>
      </varlistentry>
 
+     <varlistentry><term><option>Changelogs::URI</option> scope</term>
+	<listitem><para>
+	  Acquiring changelogs can only be done if an URI is known from where to get them.
+	  Preferable the Release file indicates this in a 'Changelogs' field. If this isn't
+	  available the Label/Origin field of the Release file is used to check if a
+	  <literal>Acquire::Changelogs::URI::Label::<replaceable>LABEL</replaceable></literal> or
+	  <literal>Acquire::Changelogs::URI::Origin::<replaceable>ORIGIN</replaceable></literal> option
+	  exists and if so this value is taken. The value in the Release file can be overridden
+	  with <literal>Acquire::Changelogs::URI::Override::Label::<replaceable>LABEL</replaceable></literal>
+	  or <literal>Acquire::Changelogs::URI::Override::Origin::<replaceable>ORIGIN</replaceable></literal>.
+
+	  The value should be a normal URI to a text file, expect that package specific data is
+	  replaced with the placeholder <literal>CHANGEPATH</literal>. The
+	  value for it is: 1. if the package is from a component (e.g. <literal>main</literal>)
+	  this is the first part otherwise it is omitted, 2. the first letter of source package name,
+	  expect if the source package name starts with '<literal>lib</literal>' in which case it will
+	  be the first four letters. 3. The complete source package name. 4. the complete name again and
+	  5. the source version.
+	  The first (if present), second, third and fourth part are separated by a slash ('<literal>/</literal>')
+	  and between the fourth and fifth part is an underscore ('<literal>_</literal>').
+
+	  The special value '<literal>no</literal>' is available for this option indicating that
+	  this source can't be used to acquire changelog files from. Another source will be tried
+	  if available in this case.
+	</para></listitem>
+    </varlistentry>
+
    </variablelist>
  </refsect1>
 
@@ -659,7 +690,7 @@ DPkg::Pre-Install-Pkgs {"/usr/sbin/dpkg-preconfigure --apt";};
 
    <para>
      The configuration item <literal>RootDir</literal> has a special
-     meaning.  If set, all paths in <literal>Dir::</literal> will be
+     meaning.  If set, all paths will be
      relative to <literal>RootDir</literal>, <emphasis>even paths that
      are specified absolutely</emphasis>.  So, for instance, if
      <literal>RootDir</literal> is set to
@@ -668,6 +699,7 @@ DPkg::Pre-Install-Pkgs {"/usr/sbin/dpkg-preconfigure --apt";};
      <filename>/var/lib/dpkg/status</filename>, then the status file
      will be looked up in
      <filename>/tmp/staging/var/lib/dpkg/status</filename>.
+     If you want to prefix only relative paths, set <literal>Dir</literal> instead.
    </para>
 
    <para>
@@ -743,10 +775,34 @@ DPkg::Pre-Install-Pkgs {"/usr/sbin/dpkg-preconfigure --apt";};
      install to the commands, one per line on the requested file descriptor, defaulting
      to standard input.</para>
 
-     <para>Version 2 of this protocol dumps more information, including the
-     protocol version, the APT configuration space and the packages, files
-     and versions being changed. Version 3 adds the architecture and <literal>MultiArch</literal>
-     flag to each version being dumped.</para>
+     <para>Version 2 of this protocol sends more information through the requested
+     file descriptor: a line with the text <literal>VERSION 2</literal>,
+     the APT configuration space, and a list of package actions with filename
+     and version information.</para>
+
+     <para>Each configuration directive line has the form
+     <literal>key=value</literal>.  Special characters (equal signs, newlines,
+     nonprintable characters, quotation marks, and percent signs in
+     <literal>key</literal> and newlines, nonprintable characters, and percent
+     signs in <literal>value</literal>) are %-encoded. Lists are represented
+     by multiple <literal>key::=value</literal> lines with the same key. The
+     configuration section ends with a blank line.</para>
+
+     <para>Package action lines consist of five fields in Version 2: old version, direction
+     of version change (&lt; for upgrades, &gt; for downgrades, = for no
+     change), new version, action. The version fields are "-" for no version
+     at all (for example when installing a package for the first time; no
+     version is treated as earlier than any real version, so that is an
+     upgrade, indicated as <literal>- &lt; 1.23.4</literal>). The action field
+     is "**CONFIGURE**" if the package is being configured, "**REMOVE**" if it
+     is being removed, or the filename of a .deb file if it is being
+     unpacked.</para>
+
+     <para>In Version 3 after each version field follows the architecture
+     of this version, which is "-" if there is no version, and a field showing
+     the MultiArch type "same", foreign", "allowed" or "none". Note that "none"
+     is an incorrect typename which is just kept to remain compatible, it
+     should be read as "no" and users are encouraged to support both.</para>
 
      <para>The version of the protocol to be used for the command
      <literal><replaceable>cmd</replaceable></literal> can be chosen by setting
diff --git a/doc/apt_preferences.5.xml b/doc/apt_preferences.5.xml
index 16e6a7aa0..79132e007 100644
--- a/doc/apt_preferences.5.xml
+++ b/doc/apt_preferences.5.xml
@@ -115,14 +115,17 @@ from archives which in their <filename>Release</filename> files are marked as "N
 
 <varlistentry>
 <term>priority 500</term>
-<listitem><simpara>to the versions that are not installed and do not belong to the target release.</simpara></listitem>
+<listitem><simpara>to the versions that do not belong to the target release.</simpara></listitem>
 </varlistentry>
 
 <varlistentry>
 <term>priority 990</term>
-<listitem><simpara>to the versions that are not installed and belong to the target release.</simpara></listitem>
+<listitem><simpara>to the versions that belong to the target release.</simpara></listitem>
 </varlistentry>
 </variablelist>
+
+The highest of those priorities whose description matches the version is assigned to the
+version.
 </para>
 
 <para>If the target release has not been specified then APT simply assigns
@@ -251,6 +254,11 @@ Pin-Priority: 500
 </programlisting>
 </listitem>
 </itemizedlist>
+
+The effect of the comma operator is similar to an "and" in logic: All
+conditions must be satisfied for the pin to match. There is one exception:
+For any type of condition (such as two "a" conditions), only the last such
+condition is checked.
 </para>
 
 </refsect2>
@@ -267,7 +275,7 @@ expression surrounded by slashes).
 
 <programlisting>
 Package: gnome* /kde/
-Pin: release n=experimental
+Pin: release a=experimental
 Pin-Priority: 500
 </programlisting>
 
@@ -339,14 +347,21 @@ only if there is no installed version of the package</simpara></listitem>
 <term>P &lt; 0</term>
 <listitem><simpara>prevents the version from being installed</simpara></listitem>
 </varlistentry>
+<varlistentry>
+<term>P = 0</term>
+<listitem><simpara>has undefined behaviour, do not use it.</simpara></listitem>
+</varlistentry>
 </variablelist>
 </para>
 
-<para>If any specific-form records match an available package version then the
-first such record determines the priority of the package version.  
-Failing that,
-if any general-form records match an available package version then the
-first such record determines the priority of the package version.</para>
+<para>
+The first specific-form record matching an available package version determines
+the priority of the package version.
+Failing that, the priority of the package is defined as the maximum of all
+priorities defined by generic-form records matching the version.
+Records defined using patterns in the Pin field other than "*" are treated like
+specific-form records.
+</para>
 
 <para>For example, suppose the APT preferences file contains the three
 records presented earlier:</para>
diff --git a/doc/dpkg-tech.dbk b/doc/dpkg-tech.dbk
index 2584cf640..f95716cf4 100644
--- a/doc/dpkg-tech.dbk
+++ b/doc/dpkg-tech.dbk
@@ -404,15 +404,7 @@ As yet unwritten. You can refer to the other manuals for now. See
 <itemizedlist>
 <listitem>
 <para>
-DPKG_NO_TSTP - if set to a non-null value, this variable causes dpkg to run a
-child shell process instead of sending itself a SIGTSTP, when the user selects
-to background the dpkg process when it asks about conffiles.
-</para>
-</listitem>
-<listitem>
-<para>
-SHELL - used to determine which shell to run in the case when DPKG_NO_TSTP
-is set.
+SHELL - used to determine which shell to run.
 </para>
 </listitem>
 <listitem>
diff --git a/doc/examples/configure-index b/doc/examples/configure-index
index ef1ae056d..1339335fa 100644
--- a/doc/examples/configure-index
+++ b/doc/examples/configure-index
@@ -117,14 +117,6 @@ APT
   // does a ExecFork)
   Keep-Fds {};
 
-  Changelogs
-  {
-    // server the provides the changelogs, the code will assume
-    // the changlogs are in the pool/ under a srcpkg_ver directory 
-    // with the name "changelog"
-    Server "http://packages.debian.org/changelogs";
-  }:
-
   // control parameters for cron jobs by /etc/cron.daily/apt
   Periodic
   {
@@ -305,6 +297,9 @@ Acquire
      "none";
      "fr";
   };
+
+  // Location of the changelogs with the placeholder CHANGEPATH (e.g. "main/a/apt/apt_1.1")
+  Changelogs::URI::Origin::Debian "http://metadata.ftp-master.debian.org/changelogs/CHANGEPATH_changelog";
 };
 
 // Directory layout
diff --git a/doc/files.dbk b/doc/files.dbk
index f513e0008..714255a53 100644
--- a/doc/files.dbk
+++ b/doc/files.dbk
@@ -198,11 +198,11 @@ installation
 <section id="s2.4"><title>Binary Package Cache (srcpkgcache.bin and pkgcache.bin)</title>
 <para>
 Please see cache.sgml for a complete description of what this file
-is. The cache file is updated whenever the contents of the lists
-directory changes. If the cache is erased, corrupted or of a non-matching
+is. The cache file is updated whenever the Packages or Release files of the lists
+directory or the dpkg status file changes. If the cache is erased, corrupted or of a non-matching
 version it will be automatically rebuilt by all of the tools that need
 it. <emphasis>srcpkgcache.bin</emphasis> contains a cache of all of the
-package files in the source list. This allows regeneration of the cache
+package, release files in the source list. In comparison to <emphasis>pkgcache.bin</emphasis>, it does not include the /var/lib/dpkg/status file. This allows regeneration of the cache
 when the status files change to use a prebuilt version for greater speed.
 </para>
 </section>
diff --git a/doc/po/apt-doc.pot b/doc/po/apt-doc.pot
index 98c2b66b5..35db9571d 100644
--- a/doc/po/apt-doc.pot
+++ b/doc/po/apt-doc.pot
@@ -531,7 +531,8 @@ msgid ""
 "<literal>list</literal> is used to display a list of packages. It supports "
 "shell pattern for matching package names and the following options: "
 "<option>--installed</option>, <option>--upgradable</option>, "
-"<option>--all-versions</option> are supported."
+"<option>--upgradeable</option>, <option>--all-versions</option> are "
+"supported."
 msgstr ""
 
 #. type: Content of: <refentry><refsect1><variablelist><varlistentry><listitem><para>
@@ -4241,7 +4242,7 @@ msgstr ""
 #, no-wrap
 msgid ""
 "Package: gnome* /kde/\n"
-"Pin: release n=experimental\n"
+"Pin: release a=experimental\n"
 "Pin-Priority: 500\n"
 msgstr ""
 
diff --git a/doc/po/de.po b/doc/po/de.po
index 4eb458d17..4f001b85e 100644
--- a/doc/po/de.po
+++ b/doc/po/de.po
@@ -662,13 +662,13 @@ msgstr ""
 msgid ""
 "<literal>list</literal> is used to display a list of packages. It supports "
 "shell pattern for matching package names and the following options: "
-"<option>--installed</option>, <option>--upgradable</option>, <option>--all-"
+"<option>--installed</option>, <option>--upgradable</option>, <option>--upgradeable</option>, <option>--all-"
 "versions</option> are supported."
 msgstr ""
 "<literal>list</literal> wird benutzt, um eine Paketliste anzuzeigen. Es "
 "unterstützt Shell-Muster zur Beschränkung auf passende Paketnamen. Die "
 "folgenden Optionen werden unterstützt: <option>--installed</option>, "
-"<option>--upgradable</option>, <option>--all-versions</option>."
+"<option>--upgradable</option>, <option>--upgradeable</option>, <option>--all-versions</option>."
 
 #. type: Content of: <refentry><refsect1><variablelist><varlistentry><listitem><para>
 #: apt.8.xml:54
@@ -6071,11 +6071,11 @@ msgstr ""
 #, no-wrap
 msgid ""
 "Package: gnome* /kde/\n"
-"Pin: release n=experimental\n"
+"Pin: release a=experimental\n"
 "Pin-Priority: 500\n"
 msgstr ""
 "Package: gnome* /kde/\n"
-"Pin: release n=experimental\n"
+"Pin: release a=experimental\n"
 "Pin-Priority: 500\n"
 
 #. type: Content of: <refentry><refsect1><refsect2><para>
diff --git a/doc/po/es.po b/doc/po/es.po
index 590ae4aeb..84dd3a127 100644
--- a/doc/po/es.po
+++ b/doc/po/es.po
@@ -36,7 +36,7 @@
 #
 msgid ""
 msgstr ""
-"Project-Id-Version: apt 0.9.7.1\n"
+"Project-Id-Version: apt-doc 1.0.5\n"
 "Report-Msgid-Bugs-To: APT Development Team <deity@lists.debian.org>\n"
 "POT-Creation-Date: 2014-11-06 09:45+0100\n"
 "PO-Revision-Date: 2014-07-04 01:31+0200\n"
@@ -742,7 +742,7 @@ msgstr ""
 msgid ""
 "<literal>list</literal> is used to display a list of packages. It supports "
 "shell pattern for matching package names and the following options: "
-"<option>--installed</option>, <option>--upgradable</option>, <option>--all-"
+"<option>--installed</option>, <option>--upgradable</option>, <option>--upgradeable</option>, <option>--all-"
 "versions</option> are supported."
 msgstr ""
 
@@ -6126,11 +6126,11 @@ msgstr ""
 #, no-wrap
 msgid ""
 "Package: gnome* /kde/\n"
-"Pin: release n=experimental\n"
+"Pin: release a=experimental\n"
 "Pin-Priority: 500\n"
 msgstr ""
 "Package: gnome* /kde/\n"
-"Pin: release n=experimental\n"
+"Pin: release a=experimental\n"
 "Pin-Priority: 500\n"
 
 #. type: Content of: <refentry><refsect1><refsect2><para>
diff --git a/doc/po/fr.po b/doc/po/fr.po
index 85869a2b7..d1e60a655 100644
--- a/doc/po/fr.po
+++ b/doc/po/fr.po
@@ -9,7 +9,7 @@
 # Jean-Pierre Giraud <jean-pierregiraud@neuf.fr>, 2014.
 msgid ""
 msgstr ""
-"Project-Id-Version: \n"
+"Project-Id-Version: apt-doc 1.0.5\n"
 "Report-Msgid-Bugs-To: APT Development Team <deity@lists.debian.org>\n"
 "POT-Creation-Date: 2015-03-09 02:17+0100\n"
 "PO-Revision-Date: 2014-11-15 17:26+0100\n"
@@ -658,7 +658,7 @@ msgstr ""
 msgid ""
 "<literal>list</literal> is used to display a list of packages. It supports "
 "shell pattern for matching package names and the following options: "
-"<option>--installed</option>, <option>--upgradable</option>, <option>--all-"
+"<option>--installed</option>, <option>--upgradable</option>, <option>--upgradeable</option>, <option>--all-"
 "versions</option> are supported."
 msgstr ""
 "La commande <literal>list</literal> est utilisée pour afficher une liste de "
@@ -6045,11 +6045,11 @@ msgstr ""
 #, no-wrap
 msgid ""
 "Package: gnome* /kde/\n"
-"Pin: release n=experimental\n"
+"Pin: release a=experimental\n"
 "Pin-Priority: 500\n"
 msgstr ""
 "Package: gnome* /kde/\n"
-"Pin: release n=experimental\n"
+"Pin: release a=experimental\n"
 "Pin-Priority: 500\n"
 
 #. type: Content of: <refentry><refsect1><refsect2><para>
diff --git a/doc/po/it.po b/doc/po/it.po
index 28e97f14b..af5fa2914 100644
--- a/doc/po/it.po
+++ b/doc/po/it.po
@@ -8,7 +8,7 @@
 # Beatrice Torracca <beatricet@libero.it>, 2012, 2014, 2015.
 msgid ""
 msgstr ""
-"Project-Id-Version: \n"
+"Project-Id-Version: apt-doc 1.0.5\n"
 "Report-Msgid-Bugs-To: APT Development Team <deity@lists.debian.org>\n"
 "POT-Creation-Date: 2015-03-10 09:48+0100\n"
 "PO-Revision-Date: 2015-01-27 14:11+0200\n"
@@ -710,13 +710,13 @@ msgstr ""
 msgid ""
 "<literal>list</literal> is used to display a list of packages. It supports "
 "shell pattern for matching package names and the following options: "
-"<option>--installed</option>, <option>--upgradable</option>, <option>--all-"
+"<option>--installed</option>, <option>--upgradable</option>, <option>--upgradeable</option>, <option>--all-"
 "versions</option> are supported."
 msgstr ""
 "<literal>list</literal> viene usato per visualizzare un elenco di pacchetti. "
 "Permette l'uso dei modelli di shell per la corrispondenza con nomi di "
 "pacchetto e sono gestite le seguenti opzioni: <option>--installed</option>, "
-"<option>--upgradable</option>, <option>--all-versions</option>."
+"<option>--upgradable</option>, <option>--upgradeable</option>, <option>--all-versions</option>."
 
 #. type: Content of: <refentry><refsect1><variablelist><varlistentry><listitem><para>
 #: apt.8.xml:54
@@ -6072,11 +6072,11 @@ msgstr ""
 #, no-wrap
 msgid ""
 "Package: gnome* /kde/\n"
-"Pin: release n=experimental\n"
+"Pin: release a=experimental\n"
 "Pin-Priority: 500\n"
 msgstr ""
 "Package: gnome* /kde/\n"
-"Pin: release n=experimental\n"
+"Pin: release a=experimental\n"
 "Pin-Priority: 500\n"
 
 #. type: Content of: <refentry><refsect1><refsect2><para>
diff --git a/doc/po/ja.po b/doc/po/ja.po
index 3e0712125..a74d2af3a 100644
--- a/doc/po/ja.po
+++ b/doc/po/ja.po
@@ -6,7 +6,7 @@
 # KURASAWA Nozomu, 2003-2006, 2009-2012.
 msgid ""
 msgstr ""
-"Project-Id-Version: apt 1.0.6\n"
+"Project-Id-Version: apt-doc 1.0.6\n"
 "Report-Msgid-Bugs-To: APT Development Team <deity@lists.debian.org>\n"
 "POT-Creation-Date: 2014-11-06 09:45+0100\n"
 "PO-Revision-Date: 2014-07-10 19:52+0900\n"
@@ -704,12 +704,12 @@ msgstr ""
 msgid ""
 "<literal>list</literal> is used to display a list of packages. It supports "
 "shell pattern for matching package names and the following options: "
-"<option>--installed</option>, <option>--upgradable</option>, <option>--all-"
+"<option>--installed</option>, <option>--upgradable</option>, <option>--upgradeable</option>, <option>--all-"
 "versions</option> are supported."
 msgstr ""
 "パッケージ一覧を表示するには <literal>list</literal> を使います。パッケージ名"
 "のマッチングにシェルパターン、そしてオプション <option>--installed</"
-"option>、 <option>--upgradable</option>、 <option>--all-versions</option> を"
+"option>、 <option>--upgradable</option>, <option>--upgradeable</option>、 <option>--all-versions</option> を"
 "サポートしています。"
 
 #. type: Content of: <refentry><refsect1><variablelist><varlistentry><listitem><para>
@@ -5812,11 +5812,11 @@ msgstr ""
 #, no-wrap
 msgid ""
 "Package: gnome* /kde/\n"
-"Pin: release n=experimental\n"
+"Pin: release a=experimental\n"
 "Pin-Priority: 500\n"
 msgstr ""
 "Package: gnome* /kde/\n"
-"Pin: release n=experimental\n"
+"Pin: release a=experimental\n"
 "Pin-Priority: 500\n"
 
 #. type: Content of: <refentry><refsect1><refsect2><para>
diff --git a/doc/po/pl.po b/doc/po/pl.po
index 481d85af6..229ac02c1 100644
--- a/doc/po/pl.po
+++ b/doc/po/pl.po
@@ -9,7 +9,7 @@
 # Robert Luberda <robert@debian.org> 2000-2004, 2010, 2012.
 msgid ""
 msgstr ""
-"Project-Id-Version: apt 0.9.7.3\n"
+"Project-Id-Version: apt-doc 1.0.5\n"
 "Report-Msgid-Bugs-To: APT Development Team <deity@lists.debian.org>\n"
 "POT-Creation-Date: 2014-11-06 09:45+0100\n"
 "PO-Revision-Date: 2014-07-04 02:13+0200\n"
@@ -700,7 +700,7 @@ msgstr ""
 msgid ""
 "<literal>list</literal> is used to display a list of packages. It supports "
 "shell pattern for matching package names and the following options: "
-"<option>--installed</option>, <option>--upgradable</option>, <option>--all-"
+"<option>--installed</option>, <option>--upgradable</option>, <option>--upgradeable</option>, <option>--all-"
 "versions</option> are supported."
 msgstr ""
 
@@ -5509,11 +5509,11 @@ msgstr ""
 #, no-wrap
 msgid ""
 "Package: gnome* /kde/\n"
-"Pin: release n=experimental\n"
+"Pin: release a=experimental\n"
 "Pin-Priority: 500\n"
 msgstr ""
 "Package: gnome* /kde/\n"
-"Pin: release n=experimental\n"
+"Pin: release a=experimental\n"
 "Pin-Priority: 500\n"
 
 #. type: Content of: <refentry><refsect1><refsect2><para>
diff --git a/doc/po/pt.po b/doc/po/pt.po
index 3aefcc2a9..35efdfe30 100644
--- a/doc/po/pt.po
+++ b/doc/po/pt.po
@@ -5,7 +5,7 @@
 # Américo Monteiro <a_monteiro@gmx.com>, 2009 - 2014.
 msgid ""
 msgstr ""
-"Project-Id-Version: apt 1.0.7\n"
+"Project-Id-Version: apt-doc 1.0.7\n"
 "Report-Msgid-Bugs-To: APT Development Team <deity@lists.debian.org>\n"
 "POT-Creation-Date: 2014-11-06 09:45+0100\n"
 "PO-Revision-Date: 2014-08-29 00:34+0100\n"
@@ -707,7 +707,7 @@ msgstr ""
 msgid ""
 "<literal>list</literal> is used to display a list of packages. It supports "
 "shell pattern for matching package names and the following options: "
-"<option>--installed</option>, <option>--upgradable</option>, <option>--all-"
+"<option>--installed</option>, <option>--upgradable</option>, <option>--upgradeable</option>, <option>--all-"
 "versions</option> are supported."
 msgstr ""
 "<literal>list</literal> é usado para mostrar uma lista de pacotes. Suporta "
@@ -6007,11 +6007,11 @@ msgstr ""
 #, no-wrap
 msgid ""
 "Package: gnome* /kde/\n"
-"Pin: release n=experimental\n"
+"Pin: release a=experimental\n"
 "Pin-Priority: 500\n"
 msgstr ""
 "Package: gnome* /kde/\n"
-"Pin: release n=experimental\n"
+"Pin: release a=experimental\n"
 "Pin-Priority: 500\n"
 
 #. type: Content of: <refentry><refsect1><refsect2><para>
diff --git a/doc/po/pt_BR.po b/doc/po/pt_BR.po
index 88b479e75..9a9f8ec56 100644
--- a/doc/po/pt_BR.po
+++ b/doc/po/pt_BR.po
@@ -7,7 +7,7 @@
 #
 msgid ""
 msgstr ""
-"Project-Id-Version: apt\n"
+"Project-Id-Version: apt-doc 1.0.5\n"
 "Report-Msgid-Bugs-To: APT Development Team <deity@lists.debian.org>\n"
 "POT-Creation-Date: 2014-11-06 09:45+0100\n"
 "PO-Revision-Date: 2004-09-20 17:02+0000\n"
@@ -523,7 +523,7 @@ msgstr ""
 msgid ""
 "<literal>list</literal> is used to display a list of packages. It supports "
 "shell pattern for matching package names and the following options: "
-"<option>--installed</option>, <option>--upgradable</option>, <option>--all-"
+"<option>--installed</option>, <option>--upgradable</option>, <option>--upgradeable</option>, <option>--all-"
 "versions</option> are supported."
 msgstr ""
 
@@ -4393,7 +4393,7 @@ msgstr ""
 #, fuzzy, no-wrap
 msgid ""
 "Package: gnome* /kde/\n"
-"Pin: release n=experimental\n"
+"Pin: release a=experimental\n"
 "Pin-Priority: 500\n"
 msgstr ""
 "<programlisting>\n"
diff --git a/doc/sources.list.5.xml b/doc/sources.list.5.xml
index da4f571b5..e27eddb0e 100644
--- a/doc/sources.list.5.xml
+++ b/doc/sources.list.5.xml
@@ -31,37 +31,99 @@
  
  <refsect1><title>Description</title>
    <para>
-   The source list <filename>/etc/apt/sources.list</filename> is designed to support
-   any number of active sources and a variety of source media. The file lists one
-   source per line, with the most preferred source listed first. The information available
-   from the configured sources is acquired by <command>apt-get update</command>
-   (or by an equivalent command from another APT front-end).
-   </para>
-   <para>
-   Each line specifying a source starts with type (e.g. <literal>deb-src</literal>)
-   followed by options and arguments for this type.
-   Individual entries cannot be continued onto a following line. Empty lines
-   are ignored, and a <literal>#</literal> character anywhere on a line marks
-   the remainder of that line as a comment.
+      The source list <filename>/etc/apt/sources.list</filename> and the the
+      files contained in <filename>/etc/apt/sources.list.d/</filename> are
+      designed to support any number of active sources and a variety of source
+      media. The files list one source per line (one line style) or contain multiline
+      stanzas defining one or more sources per stanza (deb822 style), with the
+      most preferred source listed first (in case a single version is available from more than one source). The information available from the
+      configured sources is acquired by <command>apt-get update</command> (or
+      by an equivalent command from another APT front-end).
    </para>
  </refsect1>
  
  <refsect1><title>sources.list.d</title>
-   <para>The <filename>/etc/apt/sources.list.d</filename> directory provides
-   a way to add sources.list entries in separate files.
-   The format is the same as for the regular <filename>sources.list</filename> file.
-   File names need to end with
-   <filename>.list</filename> and may only contain letters (a-z and A-Z),
-   digits (0-9), underscore (_), hyphen (-) and period (.) characters.
-   Otherwise APT will print a notice that it has ignored a file, unless that
-   file matches a pattern in the <literal>Dir::Ignore-Files-Silently</literal>
-   configuration list - in which case it will be silently ignored.</para>
+    <para>The <filename>/etc/apt/sources.list.d</filename> directory provides
+       a way to add sources.list entries in separate files.
+       Two different file formats are allowed as described in the next two sections.
+       Filenames need to have either the extension <filename>.list</filename> or
+       <filename>.sources</filename> depending on the contained format.
+       The filenames may only contain letters (a-z and A-Z),
+       digits (0-9), underscore (_), hyphen (-) and period (.) characters.
+       Otherwise APT will print a notice that it has ignored a file, unless that
+       file matches a pattern in the <literal>Dir::Ignore-Files-Silently</literal>
+       configuration list - in which case it will be silently ignored.</para>
+ </refsect1>
+
+ <refsect1><title>one line style format</title>
+    <para>
+       Files in this format have the extension <filename>.list</filename>.
+       Each line specifying a source starts with a type (e.g. <literal>deb-src</literal>)
+       followed by options and arguments for this type.
+
+       Individual entries cannot be continued onto a following line. Empty lines
+       are ignored, and a <literal>#</literal> character anywhere on a line marks
+       the remainder of that line as a comment. Consequently an entry can be
+       disabled by commenting out the entire line.
+
+       If options should be provided they are separated by spaces and all of
+       them together are enclosed by square brackets (<literal>[]</literal>)
+       included in the line after the type separated from it with a space.
+       If an option allows multiple values these are separated from each other
+       with a comma (<literal>,</literal>). An option name is separated from its
+       value(s) by a equal sign (<literal>=</literal>). Multivalue options have
+       also <literal>-=</literal> and <literal>+=</literal> as separator which
+       instead of replacing the default with the given value(s) modify the default
+       value(s) to remove or include the given values.
+   </para><para>
+       This is the traditional format and supported by all apt versions.
+       Note that not all options as described below are supported by all apt versions.
+       Note also that some older applications parsing this format on its own might not
+       expect to encounter options as they were uncommon before the introduction of
+       multi-architecture support.
+    </para>
+ </refsect1>
+
+ <refsect1><title>deb822 style format</title>
+    <para>
+       Files in this format have the extension <filename>.sources</filename>.
+       The format is similar in syntax to other files used by Debian and its
+       derivatives, like the metadata itself apt will download from the configured
+       sources or the <filename>debian/control</filename> file in a Debian source package.
+
+       Individual entries are separated by an empty line, additional empty
+       lines are ignored, and a <literal>#</literal> character at the start of
+       the line marks the entire line as a comment. An entry can hence be
+       disabled by commenting out each line belonging to the stanza, but it is
+       usually easier to add the field "Enabled: no" to the stanza to disable
+       the entry. Removing the field or setting it to yes reenables it.
+
+       Options have the same syntax as every other field: A fieldname separated by
+       a colon (<literal>:</literal>) and optionally spaces from its value(s).
+       Note especially that multiple values are separated by spaces, not by
+       commas as in the one line format. Multivalue fields like <literal>Architectures</literal>
+       also have <literal>Architectures-Add</literal> and <literal>Architectures-Remove</literal>
+       to modify the default value rather than replacing it.
+   </para><para>
+       This is a new format supported by apt itself since version 1.1. Previous
+       versions ignore such files with a notice message as described earlier.
+       It is intended to make this format gradually the default format and
+       deprecating the previously described one line style format as it is
+       easier to create, extend and modify by humans and machines alike
+       especially if a lot of sources and/or options are involved.
+
+       Developers who are working with and/or parsing apt sources are highly
+       encouraged to add support for this format and to contact the APT team
+       to coordinate and share this work. Users can freely adopt this format
+       already, but could encounter problems with software not supporting
+       the format yet.
+    </para>
  </refsect1>
 
- <refsect1><title>The deb and deb-src types</title>
+ <refsect1><title>The deb and deb-src types: General Format</title>
    <para>The <literal>deb</literal> type references a typical two-level Debian
    archive, <filename>distribution/component</filename>. The
-   <literal>distribution</literal> is generally an archive name like
+   <literal>distribution</literal> is generally a suite name like
    <literal>stable</literal> or <literal>testing</literal> or a codename like
    <literal>&stable-codename;</literal> or <literal>&testing-codename;</literal>
    while component is one of <literal>main</literal>, <literal>contrib</literal> or
@@ -70,42 +132,33 @@
    code in the same form as the <literal>deb</literal> type.
    A <literal>deb-src</literal> line is required to fetch source indexes.</para>
 
-   <para>The format for a <filename>sources.list</filename> entry using the
+   <para>The format for two one line style entries using the
    <literal>deb</literal> and <literal>deb-src</literal> types is:</para>
 
-   <literallayout>deb [ options ] uri suite [component1] [component2] [...]</literallayout>
+   <literallayout>deb [ option1=value1 option2=value2 ] uri suite [component1] [component2] [...]
+deb-src [ option1=value1 option2=value2 ] uri suite [component1] [component2] [...]</literallayout>
 
-   <para>Alternatively a rfc822 style format is also supported:
+   <para>Alternatively the equivalent entry in deb822 style looks like this:
    <literallayout>
      Types: deb deb-src
-     URIs: http://example.com
-     Suites: stable testing
-     Sections: component1 component2
-     Description: short
-      long long long
-     [option1]: [option1-value]
-
-     Types: deb
-     URIs: http://another.example.com
-     Suites: experimental
-     Sections: component1 component2
-     Enabled: no
-     Description: short
-      long long long
-     [option1]: [option1-value]
+     URIs: uri
+     Suites: suite
+     Components: [component1] [component2] [...]
+     option1: value1
+     option2: value2
    </literallayout>
    </para>
 
    <para>The URI for the <literal>deb</literal> type must specify the base of the
-   Debian distribution, from which APT will find the information it needs. 
-   <literal>suite</literal> can specify an exact path, in which case the 
+   Debian distribution, from which APT will find the information it needs.
+   <literal>suite</literal> can specify an exact path, in which case the
    components must be omitted and <literal>suite</literal> must end with
    a slash (<literal>/</literal>). This is useful for the case when only a
-   particular sub-section of the archive denoted by the URI is of interest.
+   particular sub-directory of the archive denoted by the URI is of interest.
    If <literal>suite</literal> does not specify an exact path, at least
    one <literal>component</literal> must be present.</para>
 
-   <para><literal>suite</literal> may also contain a variable, 
+   <para><literal>suite</literal> may also contain a variable,
    <literal>$(ARCH)</literal>
    which expands to the Debian architecture (such as <literal>amd64</literal> or
    <literal>armel</literal>) used on the system. This permits architecture-independent
@@ -113,51 +166,123 @@
    of interest when specifying an exact path, <literal>APT</literal> will
    automatically generate a URI with the current architecture otherwise.</para>
 
-   <para>In the traditional style sources.list format since only one
-   distribution can be specified per line it may be necessary to have
-   multiple lines for the same URI, if a subset of all available
-   distributions or components at that location is desired.  APT will
-   sort the URI list after it has generated a complete set internally,
-   and will collapse multiple references to the same Internet host,
-   for instance, into a single connection, so that it does not
-   inefficiently establish an FTP connection, close it, do something
-   else, and then re-establish a connection to that same host. This
-   feature is useful for accessing busy FTP sites with limits on the
-   number of simultaneous anonymous users. APT also parallelizes
-   connections to different hosts to more effectively deal with sites
-   with low bandwidth.</para>
-
-   <para><literal>options</literal> is always optional and needs to be surrounded by
-   square brackets. It can consist of multiple settings in the form
-   <literal><replaceable>setting</replaceable>=<replaceable>value</replaceable></literal>.
-   Multiple settings are separated by spaces. The following settings are supported by APT
-   (note however that unsupported settings will be ignored silently):
-   <itemizedlist>
-   <listitem><para><literal>arch=<replaceable>arch1</replaceable>,<replaceable>arch2</replaceable>,…</literal>
-   can be used to specify for which architectures information should
-   be downloaded. If this option is not set all architectures defined by the
-   <literal>APT::Architectures</literal> option will be downloaded.</para></listitem>
-   <listitem><para><literal>arch+=<replaceable>arch1</replaceable>,<replaceable>arch2</replaceable>,…</literal>
-   and <literal>arch-=<replaceable>arch1</replaceable>,<replaceable>arch2</replaceable>,…</literal>
-   which can be used to add/remove architectures from the set which will be downloaded.</para></listitem>
-   <listitem><para><literal>trusted=yes</literal> can be set to indicate that packages
-   from this source are always authenticated even if the <filename>Release</filename> file
-   is not signed or the signature can't be checked. This disables parts of &apt-secure;
-   and should therefore only be used in a local and trusted context. <literal>trusted=no</literal>
-   is the opposite which handles even correctly authenticated sources as not authenticated.</para></listitem>
-   </itemizedlist></para>
+   <para>Especially in the one line style format since only one distribution
+      can be specified per line it may be necessary to have multiple lines for
+      the same URI, if a subset of all available distributions or components at
+      that location is desired.  APT will sort the URI list after it has
+      generated a complete set internally, and will collapse multiple
+      references to the same Internet host, for instance, into a single
+      connection, so that it does not inefficiently establish a
+      connection, close it, do something else, and then re-establish a
+      connection to that same host. APT also parallelizes connections to
+      different hosts to more effectively deal with sites with low
+      bandwidth.</para>
 
    <para>It is important to list sources in order of preference, with the most
    preferred source listed first. Typically this will result in sorting
    by speed from fastest to slowest (CD-ROM followed by hosts on a local
    network, followed by distant Internet hosts, for example).</para>
 
-   <para>Some examples:</para>
-   <literallayout>
-deb http://ftp.debian.org/debian &stable-codename; main contrib non-free
-deb http://security.debian.org/ &stable-codename;/updates main contrib non-free
-   </literallayout>
+   <para>As an example, the sources for your distribution could look like this
+      in one line style format:
+      <literallayout>&sourceslist-list-format;</literallayout> or like this in
+      deb822 style format:
+      <literallayout>&sourceslist-sources-format;</literallayout></para>
+ </refsect1>
+
+ <refsect1><title>The deb and deb-src types: Options</title>
+    <para>Each source entry can have options specified modifying which and how
+       the source is accessed and data acquired from it. Format, syntax and names
+       of the options varies between the two formats one line and deb822 style
+       as described, but they have both the same options available. For simplicity
+       we list the deb822 fieldname and provide the one line name in brackets.
+       Remember that beside setting multivalue options explicitly, there is also
+       the option to modify them based on the default, but we aren't listing those
+       names explicitly here. Unsupported options are silently ignored by all
+       APT versions.
+
+       <itemizedlist>
+	  <listitem><para><option>Architectures</option>
+		(<option>arch</option>) is a multivalue option defining for
+		which architectures information should be downloaded. If this
+		option isn't set the default is all architectures as defined by
+		the <option>APT::Architectures</option> config option.
+	  </para></listitem>
 
+	  <listitem><para><option>Languages</option>
+		(<option>lang</option>) is a multivalue option defining for
+		which languages information like translated package
+		descriptions should be downloaded.  If this option isn't set
+		the default is all languages as defined by the
+		<option>Acquire::Languages</option> config option.
+	  </para></listitem>
+
+	  <listitem><para><option>Targets</option>
+		(<option>target</option>) is a multivalue option defining
+		which download targets apt will try to acquire from this
+		source. If not specified, the default set is defined by the
+		<option>Acquire::IndexTargets</option> configuration scope.
+	  </para></listitem>
+       </itemizedlist>
+
+       Further more, there are options which if set effect
+       <emphasis>all</emphasis> sources with the same URI and Suite, so they
+       have to be set on all such entries and can not be varied between
+       different components. APT will try to detect and error out on such
+       anomalies.
+
+       <itemizedlist>
+	  <listitem><para><option>Signed-By</option> (<option>signed-by</option>)
+		is either an absolute path to a keyring file (has to be
+		accessible and readable for the <literal>_apt</literal> user,
+		so ensure everyone has read-permissions on the file) or a
+		fingerprint of a key in either the
+		<filename>trusted.gpg</filename> keyring or in one of the
+		keyrings in the <filename>trusted.gpg.d/</filename> directory
+		(see <command>apt-key fingerprint</command>). If the option is
+		set only the key(s) in this keyring or only the key with this
+		fingerprint is used for the &apt-secure; verification of this
+		repository. Otherwise all keys in the trusted keyrings are
+		considered valid signers for this repository.
+	  </para></listitem>
+
+	  <listitem><para><option>Check-Valid-Until</option> (<option>check-valid-until</option>)
+		is a yes/no value which controls if APT should try to detect
+		replay attacks. A repository creator can declare until then the
+		data provided in the repository should be considered valid and
+		if this time is reached, but no new data is provided the data
+		is considered expired and an error is raised.  Beside
+		increasing security as a malicious attacker can't sent old data
+		forever denying a user to be able to upgrade to a new version,
+		this also helps users identify mirrors which are no longer
+		updated. Some repositories like historic archives aren't
+		updated anymore by design through, so this check can be
+		disabled by setting this option to <literal>no</literal>.
+		Defaults to the value of configuration option
+		<option>Acquire::Check-Valid-Until</option> which itself
+		defaults to <literal>yes</literal>.
+	  </para></listitem>
+
+	  <listitem><para><option>Valid-Until-Min</option>
+		(<option>check-valid-min</option>) and
+		<option>Valid-Until-Max</option>
+		(<option>valid-until-max</option>) can be used to raise or
+		lower the time period in seconds in which the data from this
+		repository is considered valid. -Max can be especially useful
+		if the repository provides no Valid-Until field on its Release
+		file to set your own value, while -Min can be used to increase
+		the valid time on seldomly updated (local) mirrors of a more
+		frequently updated but less accessible archive (which is in the
+		sources.list as well) instead of disabling the check entirely.
+		Default to the value of the configuration options
+		<option>Acquire::Min-ValidTime</option> and
+		<option>Acquire::Max-ValidTime</option> which are both unset by
+		default.
+	  </para></listitem>
+
+       </itemizedlist>
+
+    </para>
  </refsect1>
 
  <refsect1><title>URI specification</title>
@@ -231,34 +356,70 @@ deb http://security.debian.org/ &stable-codename;/updates main contrib non-free
  </refsect1>
  
  <refsect1><title>Examples</title>
-   <para>Uses the archive stored locally (or NFS mounted) at /home/jason/debian
+   <para>Uses the archive stored locally (or NFS mounted) at /home/apt/debian
    for stable/main, stable/contrib, and stable/non-free.</para>
-   <literallayout>deb file:/home/jason/debian stable main contrib non-free</literallayout>
+   <literallayout>deb file:/home/apt/debian stable main contrib non-free</literallayout>
+   <literallayout>Types: deb
+URIs: file:/home/apt/debian
+Suites: stable
+Components: main contrib non-free</literallayout>
 
    <para>As above, except this uses the unstable (development) distribution.</para>
-   <literallayout>deb file:/home/jason/debian unstable main contrib non-free</literallayout>
+   <literallayout>deb file:/home/apt/debian unstable main contrib non-free</literallayout>
+   <literallayout>Types: deb
+URIs: file:/home/apt/debian
+Suites: unstable
+Components: main contrib non-free</literallayout>
 
    <para>Source line for the above</para>
-   <literallayout>deb-src file:/home/jason/debian unstable main contrib non-free</literallayout>
+   <literallayout>deb-src file:/home/apt/debian unstable main contrib non-free</literallayout>
+   <literallayout>Types: deb-src
+URIs: file:/home/apt/debian
+Suites: unstable
+Components: main contrib non-free</literallayout>
+
 
    <para>The first line gets package information for the architectures in <literal>APT::Architectures</literal>
    while the second always retrieves <literal>amd64</literal> and <literal>armel</literal>.</para>
-   <literallayout>deb http://ftp.debian.org/debian &stable-codename; main
-deb [ arch=amd64,armel ] http://ftp.debian.org/debian &stable-codename; main</literallayout>
+   <literallayout>deb http://httpredir.debian.org/debian &stable-codename; main
+deb [ arch=amd64,armel ] http://httpredir.debian.org/debian &stable-codename; main</literallayout>
+   <literallayout>Types: deb
+URIs: http://httpredir.debian.org/debian
+Suites: &stable-codename;
+Components: main
+
+Types: deb
+URIs: http://httpredir.debian.org/debian
+Suites: &stable-codename;
+Components: main
+Architectures: amd64 armel
+</literallayout>
 
    <para>Uses HTTP to access the archive at archive.debian.org, and uses only
    the hamm/main area.</para>
    <literallayout>deb http://archive.debian.org/debian-archive hamm main</literallayout>
+   <literallayout>Types: deb
+URIs: http://archive.debian.org/debian-archive
+Suites: hamm
+Components: main</literallayout>
 
    <para>Uses FTP to access the archive at ftp.debian.org, under the debian
    directory, and uses only the &stable-codename;/contrib area.</para>
    <literallayout>deb ftp://ftp.debian.org/debian &stable-codename; contrib</literallayout>
+   <literallayout>Types: deb
+URIs: ftp://ftp.debian.org/debian
+Suites: &stable-codename;
+Components: contrib</literallayout>
 
    <para>Uses FTP to access the archive at ftp.debian.org, under the debian
    directory, and uses only the unstable/contrib area. If this line appears as
    well as the one in the previous example in <filename>sources.list</filename>
    a single FTP session will be used for both resource lines.</para>
    <literallayout>deb ftp://ftp.debian.org/debian unstable contrib</literallayout>
+   <literallayout>Types: deb
+URIs: ftp://ftp.debian.org/debian
+Suites: unstable
+Components: contrib</literallayout>
 
    <para>Uses HTTP to access the archive at ftp.tlh.debian.org, under the
    universe directory, and uses only files found under
@@ -268,15 +429,32 @@ deb [ arch=amd64,armel ] http://ftp.debian.org/debian &stable-codename; main</li
    illustrates how to use the substitution variable; official debian
    archives are not structured like this]
    <literallayout>deb http://ftp.tlh.debian.org/universe unstable/binary-$(ARCH)/</literallayout>
+   <literallayout>Types: deb
+URIs: http://ftp.tlh.debian.org/universe
+Suites: unstable/binary-$(ARCH)/</literallayout>
    </para>
+
+   <para>Uses HTTP to get binary packages as well as sources from the stable, testing and unstable
+      suites and the components main and contrib.</para>
+   <literallayout>deb http://httpredir.debian.org/debian stable main contrib
+deb-src http://httpredir.debian.org/debian stable main contrib
+deb http://httpredir.debian.org/debian testing main contrib
+deb-src http://httpredir.debian.org/debian testing main contrib
+deb http://httpredir.debian.org/debian unstable main contrib
+deb-src http://httpredir.debian.org/debian unstable main contrib</literallayout>
+   <literallayout>Types: deb deb-src
+URIs: http://httpredir.debian.org/debian
+Suites: stable testing unstable
+Components: main contrib
+</literallayout>
+
  </refsect1>
- 
+
  <refsect1><title>See Also</title>
-   <para>&apt-cache; &apt-conf;
+   <para>&apt-get;, &apt-conf;
    </para>
  </refsect1>
 
  &manbugs;
- 
-</refentry>
 
+</refentry>