summaryrefslogtreecommitdiff
path: root/doc/files.sgml
diff options
context:
space:
mode:
Diffstat (limited to 'doc/files.sgml')
-rw-r--r--doc/files.sgml491
1 files changed, 491 insertions, 0 deletions
diff --git a/doc/files.sgml b/doc/files.sgml
new file mode 100644
index 000000000..d98d6f68a
--- /dev/null
+++ b/doc/files.sgml
@@ -0,0 +1,491 @@
+<!doctype debiandoc system>
+<!-- -*- mode: sgml; mode: fold -*- -->
+<book>
+<title>APT Files</title>
+
+<author>Jason Gunthorpe <email>jgg@debian.org</email></author>
+<version>$Id: files.sgml,v 1.1 1998/07/02 02:58:12 jgg Exp $</version>
+
+<abstract>
+This document describes the complete implementation and format of the
+installed APT directory structure. It also serves as guide to how APT
+views the Debian archive.
+</abstract>
+
+<copyright>
+Copyright &copy; Jason Gunthorpe, 1998.
+<p>
+"APT" and this document are free software; you can redistribute them and/or
+modify them under the terms of the GNU General Public License as published
+by the Free Software Foundation; either version 2 of the License, or (at your
+option) any later version.
+
+<p>
+For more details, on Debian GNU/Linux systems, see the file
+/usr/doc/copyright/GPL for the full license.
+</copyright>
+
+<toc sect>
+
+<chapt>Introduction
+<!-- General {{{ -->
+<!-- ===================================================================== -->
+<sect>General
+
+<p>
+This document serves two purposes. The first is to document the installed
+directory structure and the format and purpose of each file. The second
+purpose is to document how APT views the Debian archive and deals with
+multiple package files.
+
+<p>
+The var directory structure is as follows:
+<example>
+ /var/state/apt/
+ lists/
+ partial/
+ xstatus
+ /var/cache/apt/
+ pkgcache.bin
+ srcpkgcache.bin
+ archives/
+ partial/
+ /etc/apt/
+ sources.list
+ cdromdevs.list
+ /usr/lib/apt/
+ methods/
+ cdrom
+ ftp
+ http
+</example>
+
+<p>
+As is specified in the FHS 2.0 /var/state/apt is used for application
+data that is not expected to be user modified. /var/cache/apt is used
+for regeneratable data and is where the package cache and downloaded .debs
+go.
+</sect>
+ <!-- }}} -->
+
+<chapt>Files
+<!-- Distribution Source List {{{ -->
+<!-- ===================================================================== -->
+<sect>Distribution Source list (sources.list)
+
+<p>
+The distribution source list is used to locate archives of the debian
+distribution. It is designed to support any number of active sources and to
+support a mix of source media. The file lists one source per line, with the
+fastest source listed first. The format of each line is:
+
+<p>
+<var>type ui args</var>
+
+<p>
+The first item, <var>type</var>, indicates the format for the remainder
+of the line. It is designed to indicate the structure of the distribution
+the line is talking about. Currently the only defined value is <em>deb</em>
+which indicates a standard debian archive with a dists dir.
+
+<sect1>The deb Type
+ <p>
+ The <em>deb</em> type is to be a typical two level debian distributions,
+ dist/<var>distribution</var>/<var>component</var>. Typically distribution
+ is one of stable, unstable or frozen while component is one of main,
+ contrib, non-free or non-us. The format for the deb line is as follows:
+
+ <p>
+ deb <var>uri</var> <var>distribution</var> <var>compontent</var>
+ [<var>component</var> ...]
+
+ <p>
+ <var>uri</var> for the <em>deb</em> type must specify the base of the
+ debian distribution. APT will automatically generate the proper longer
+ URIs to get the information it needs. <var>distribution</var> can specify
+ an exact path, in this case the components must be omitted and
+ <var>distribution</var> must end in a slash.
+
+ <p>
+ Since only one distribution can be specified per deb line it may be
+ necessary to list a number of deb lines for the same URI. APT will
+ sort the URI list after it has generated a complete set to allow
+ connection reuse. It is important to order things in the sourcelist
+ from most prefered to least prefered (fastest to slowest).
+</sect1>
+
+<sect1>URI specification
+<p>
+URIs in the source list support a large number of access schemes.
+
+<taglist>
+<tag>cdrom<item>
+ The cdrom scheme is special in that If Modifed Since queries are never
+ performed and that APT knows how to match a cdrom to the name it
+ was given when first inserted. It does this by examining the date
+ and size of the package file. APT also knows all of the possible
+ prefix paths for the cdrom drives and that the user should be prompted
+ to insert a CD if it cannot be found. The path is relative to an
+ arbitary mount point (of APT's choosing) and must not start with a
+ slash. The first pathname component is the given name and is purely
+ descriptive and of the users choice. However, if a file in the root of
+ the cdrom is called 'cdname' its contents will be used instead of
+ prompting. The name serves as a tag for the cdrom and should be unique.
+ APT will track the CDROM's based on their tag and package file
+ properties.
+ <example>
+ cdrom:Debian 1.3/debian
+ </example>
+
+<tag>http<item>
+ This scheme specifies a HTTP server for the debian archive. HTTP is prefered
+ over FTP because If Modified Since queries against the Package file are
+ possible. Newer HTTP protcols may even support reget which would make
+ http the protocol of choice.
+ <example>
+ http://www.debian.org/archive
+ </example>
+
+<tag>ftp<item>
+ This scheme specifies a FTP connection to the server. FTP is limited because
+ there is no support for IMS and is hard to proxy over firewalls.
+ <example>
+ ftp://ftp.debian.org/debian
+ </example>
+
+<tag>file<item>
+ The file scheme allows an arbitary directory in the file system to be
+ considered as a debian archive. This is usefull for NFS mounts and
+ local mirrors/archives.
+ <example>
+ file:/var/debian
+ </example>
+
+<tag>mirror<item>
+ The mirror scheme is special in that it does not specify the location of a
+ debian archive but specifies the location of a list of mirrors to use
+ to access the archive. Some technique will be used to determine the
+ best choice for a mirror. The mirror file is specified in the Mirror File
+ section. If/when URIs take off they should obsolete this field.
+ <example>
+ mirror:http://www.debian.org/archivemirrors
+ </example>
+
+<tag>smb<item>
+ A possible future expansion may be to have direct support for smb (Samba
+ servers).
+ <example>
+ smb://ftp.kernel.org/pub/mirrors/debian
+ </example>
+</taglist>
+</sect1>
+
+<sect1>Hashing the URI
+<p>
+All permanent information aquired from any of the sources is stored in the
+lists directory. Thus, there must be a way to relate the filename in the
+lists directory to a line in the sourcelist. To simplify things this is
+done by quoting the URI and treating ='s as quoteable characters and
+converting / to =. The URI spec says this is done by converting a
+sensitive character into %xx where xx is the hexadecimal representation
+from the ascii character set. Examples:
+
+<example>
+http://www.debian.org/archive/dists/stable/binary-i386/Packages
+/var/state/apt/lists/www.debian.org=archive=dists=stable=binary-i386=Packages
+
+cdrom:Debian 1.3/debian/Packages
+/var/state/apt/info/Debian%201.3=debian=Packages
+</example>
+
+<p>
+The other alternative that was considered was to use a deep directory
+structure but this poses two problems, it makes it very difficult to prune
+directories back when sources are no longer used and complicates the handling
+of the partial directory. This gives a very simple way to deal with all
+of the situations that can arise. The equals sign was choosen on the
+suggestion of Manoj because it is very infrequently used in filenames.
+Also note that the same rules described in the <em>Archive Directory</>
+section regarding the partial sub dir apply here as well.
+</sect1>
+
+</sect>
+ <!-- }}} -->
+<!-- Extra Status {{{ -->
+<!-- ===================================================================== -->
+<sect>Extra Status File (xstatus)
+
+<p>
+The extra status file serves the same purpose as the normal dpkg status file
+(/var/lib/dpkg/status) except that it stores information unique to diety.
+This includes the autoflag, target distribution and version and any other
+uniqe features that come up over time. It duplicates nothing from the normal
+dpkg status file. Please see other APT documentation for a discussion
+of the exact internal behavior of these fields. The Package field is
+placed directly before the new fields to indicate which package they
+apply to. The new fields are as follows:
+
+<taglist>
+<tag>X-Auto<item>
+ The Auto flag can be Yes or No and controls whether the package is in
+ auto mode.
+
+<tag>X-TargetDist<item>
+ The TargetDist item indicates which distribution versions are offered for
+ installation from. It should be stable, unstable or frozen.
+
+<tag>X-TargetVersion<item>
+ The target version item is set if the user selects a specific version, it
+ overrides the TargetDist selection if both are present.
+</taglist>
+</sect>
+ <!-- }}} -->
+<!-- Binary Package Cache {{{ -->
+<!-- ===================================================================== -->
+<sect>Binary Package Cache (pkgcache.bin)
+
+<p>
+Please see cache.sgml for a complete description of what this file is. The
+cache file is updated whenever the contents of the lists directory changes.
+If the cache is erased, corrupted or of a non-matching version it will
+be automatically rebuilt by all of the tools that need it.
+<em>srcpkgcache.bin</> contains a cache of all of the package files in the
+source list. This allows regeneration of the cache when the status files
+change to use a prebuilt version for greater speed.
+</sect>
+ <!-- }}} -->
+<!-- Downloads Directory {{{ -->
+<!-- ===================================================================== -->
+<sect>Downloads Directory (archives)
+
+<p>
+The archives directory is where all downloaded .deb archives go. When the
+file transfer is initiated the deb is placed in partial. Once the file
+is fully downloaded and its MD5 hash and size are verifitied it is moved
+from partial into archives/. Any files found in archives/ can be assumed
+to be verified.
+
+<p>
+No dirctory structure is transfered from the receiving site and all .deb
+file names conform to debian conventions. No short (msdos) filename should
+be placed in archives. If the need arises .debs should be unpacked, scanned
+and renamed to their correct internal names. This is mostly to prevent
+file name conflicts but other programs may depend on this if convenient.
+Downloaded .debs must be found in one of the package lists with an exact
+name + version match..
+</sect>
+ <!-- }}} -->
+<!-- The Methods Directory {{{ -->
+<!-- ===================================================================== -->
+<sect> The Methods Directory (/usr/lib/apt/methods)
+
+<p>
+Like dselect, APT will support plugable acquisition methods to complement
+its internaly supported methods. The files in
+this directory are execultables named after the URI type. APT will
+sort the required URIs and spawn these programs giving a full sorted, quoted
+list of URIs.
+
+<p>
+The interface is simple, the program will be given a list
+of URIs on the command line. The URIs will be in pairs, the first
+being the actual URI and the second being the filename to write the data to.
+The current directory will be set properly by APT and it is
+expected the method will put files relative to the current directory.
+The output of these programs is strictly speficied. The programs must accept
+nothing from stdin (stdin will be an invalid fd) and they must output
+status information to stdout according to the format below.
+Stderr will be redirected to the logging facility.
+
+<p>
+Each line sent to stdout must be a line that has a single letter and a
+space. Strings after the first letter do not need quoting, they are taken
+as is till the end of the line. The tag letters, listed in expected order,
+is as follows:
+
+<taglist>
+
+<tag>F - Change URI<item>
+This specifies a change in URI. All information after this will be applied
+to the new URI. When the URI is changed it is assumed that the old URI has
+completed unless an error is set. The format is <var>F URI</>
+
+<tag>S - Object Size<item>
+This specifies the expected size of the object. APT will use this to
+compute percent done figures. If it is not sent then a kilobyte meter
+will be used instead of a percent display. The foramat is <var>S INTEGER</>
+
+<tag>E - Error Information<item>
+Exactly one line of error information can be set for each URI. The
+information will be summarized for the user. If an E tag is send before
+any F tags then the error is assumed to be a fatal method error and all URI
+fetches for that method are aborted with that error string. The format
+is <var>E String</>
+
+<tag>I - Informative progress information<item>
+The I tag allows the method to specify the status of the connection.
+Typically the GUI will show the last recieved I line. The format is
+<var>I String</> As a general rule an I tag should be ommitted before a
+lengthy operation only. Things that always take a short period are not
+suited for I tags. I tags should change wnenever the methods state changes.
+Some standard forms, in order of occurance, are <var>Connecting to SITE</>,
+<var>Connecting to SITE (1.1.1.1)</>, <var>Waiting for file</>,
+<var>Authenticating</>, <var>Downloading</>, <var>Resuming (size)</>,
+<var>Computing MD5</> <var>I</> lines should never print out information that
+APT is already aware of, such as file names.
+
+<tag>R - Set final path<item>
+The R tag allows the method to tell APT that the file is present in the
+local file system. APT might copy it into a the download directory. The format
+is <var>R String</>
+
+<tag>M - MD5Sum of the file<item>
+The method is expected to compute the md5 hash on the fly as the download
+progresses. The final md5 of the file is to be output when the file is
+completed. If the md5 is not output it will not be checked! Some methods
+such as the file method will not check md5's because they are most
+commonly used on mirrors or local CD-ROM's, a paranoid option may be
+provided in future to force checking. The format is <var>M MD5-String</>
+
+<tag>L - Log output<item>
+This tag indicates a string that should be dumped to some log file. The
+string is for debugging and is not ment to be seen by the user. The format
+is <var>L String</> Log things should only be used in a completed method
+if they have special relavence to what is happening.
+</taglist>
+
+<p>
+APT monitors the progress of the transfer by watching the file size. This
+means the method must not create any temp files and must use a fairly small
+buffer. The method is also responsible for If-Modified-Since (IMS) queries
+for the object. It should check ../outputname to get the time stamp but not
+size. The size may be different because the file was uncompressed after
+it was transfed. A method must <em>never</> change the file in .., it may
+only change the output file in the current directory.
+
+<p>
+The APT 'http' program is the reference implementation of this specification,
+it implements all of the features a method is expected to do.
+</sect>
+ <!-- }}} -->
+<!-- The Mirror List {{{ -->
+<!-- ===================================================================== -->
+<sect> The Mirror List
+
+<p>
+The mirror list is stored on the primary debian web server (www.debian.org)
+and contains a machine readable list of all known debian mirrors. The mirror
+URI type will cause this list to be downloaded and considered. It has the
+same form as the source list. When the source list specifies mirror
+as the target the mirror list is scanned to find the nescessary parts for
+the requested distributions and components. This means the user could
+have a line like:
+
+<var>deb mirror:http://www.debian.org/mirrorlist stable main non-us</var>
+
+which would likely cause APT to choose two separate sites to download from,
+one for main and another for non-us.
+
+<p>
+Some form of network measurement will have to be used to gauge performance
+of each of the mirrors. This will be discussed later, initial versions
+will use the first found URI.
+</sect>
+ <!-- }}} -->
+<!-- The Release File {{{ -->
+<!-- ===================================================================== -->
+<sect> The Release File
+
+<p>
+This file plays and important role in how APT presents the archive to the
+user. Its main purpose is to present a descriptive name for the source
+of each version of each package. It also is used to detect when new versions
+of debian are released. It augments the package file it is associated with
+by providing meta information about the entire archive which the Packages
+file describes.
+
+<p>
+The full name of the distribution for presentation to the user is formed
+as 'label version archive', with a possible extended name being
+'label version archive component'.
+
+<p>
+The file is formed as the package file (RFC-822) with the following tags
+defined:
+
+<taglist>
+<tag>Archive<item>
+This is the common name we give our archives, such as <em>stable</> or
+<em>unstable</>.
+
+<tag>Component<item>
+Referes to the sub-component of the archive, <em>main</>, <em>contrib</>
+etc.
+
+<tag>Version<item>
+This is a version string with the same properties as in the Packages file.
+It represents the release level of the archive.
+
+<tag>Origin<item>
+This specifies who is providing this archive. In the case of Debian the
+string will read 'Debian'. Other providers may use their own string
+
+<tag>Label<item>
+This carries the encompassing name of the distribution. For Debian proper
+this field reads 'Debian'. For derived distributions it should contain their
+proper name.
+
+<tag>Architecture<item>
+When the archive has packages for a single architecture then the Architecture
+is listed here. If a mixed set of systems are represented then this should
+contain the keyword <em>mixed</em>.
+
+<tag>NotAutomatic<item>
+A Yes/No flag indicating that the archive is extremely unstable and its
+version's should never be automatically selected. This is to be used by
+experimental.
+
+<tag>Description<item>
+Description is used to describe the release. For instance experimental would
+contain a warning that the packages have problems.
+</taglist>
+
+<p>
+The location of the Release file in the archive is very important, it must
+be located in the same location as the packages file so that it can be
+located in all situations. The following is an example for the current stable
+release, 1.3.1r6
+
+<example>
+Archive: stable
+Compontent: main
+Version: 1.3.1r6
+Origin: Debian
+Label: Debian
+Architecture: i386
+</example>
+
+This is an example of experimental,
+<example>
+Archive: experimental
+Version: 0
+Origin: Debian
+Label: Debian
+Architecture: mixed
+NotAutomatic: Yes
+</example>
+
+And unstable,
+<example>
+Archive: unstable
+Compontent: main
+Version: 2.1
+Origin: Debian
+Label: Debian
+Architecture: i386
+</example>
+
+</sect>
+ <!-- }}} -->
+
+</book>