diff options
Diffstat (limited to 'doc/apt-ftparchive.1.xml')
| -rw-r--r-- | doc/apt-ftparchive.1.xml | 627 |
1 files changed, 627 insertions, 0 deletions
diff --git a/doc/apt-ftparchive.1.xml b/doc/apt-ftparchive.1.xml new file mode 100644 index 000000000..0a9305132 --- /dev/null +++ b/doc/apt-ftparchive.1.xml @@ -0,0 +1,627 @@ +<?xml version="1.0" encoding="utf-8" standalone="no"?> +<!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" + "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [ +<!ENTITY % aptent SYSTEM "apt.ent"> %aptent; +<!ENTITY % aptverbatiment SYSTEM "apt-verbatim.ent"> %aptverbatiment; +<!ENTITY % aptvendor SYSTEM "apt-vendor.ent"> %aptvendor; +]> + +<refentry> + + <refentryinfo> + &apt-author.jgunthorpe; + &apt-author.team; + &apt-email; + &apt-product; + <!-- The last update date --> + <date>2016-11-25T00:00:00Z</date> + </refentryinfo> + + <refmeta> + <refentrytitle>apt-ftparchive</refentrytitle> + <manvolnum>1</manvolnum> + <refmiscinfo class="manual">APT</refmiscinfo> + </refmeta> + + <!-- Man page title --> + <refnamediv> + <refname>apt-ftparchive</refname> + <refpurpose>Utility to generate index files</refpurpose> + </refnamediv> + + &synopsis-command-apt-ftparchive; + + <refsect1><title>Description</title> + <para><command>apt-ftparchive</command> is the command line tool that generates the index + files that APT uses to access a distribution source. The index files should + be generated on the origin site based on the content of that site.</para> + + <para><command>apt-ftparchive</command> is a superset of the &dpkg-scanpackages; program, + incorporating its entire functionality via the <literal>packages</literal> command. + It also contains a contents file generator, <literal>contents</literal>, and an + elaborate means to 'script' the generation process for a complete + archive.</para> + + <para>Internally <command>apt-ftparchive</command> can make use of binary databases to + cache the contents of a .deb file and it does not rely on any external + programs aside from &gzip;. When doing a full generate it automatically + performs file-change checks and builds the desired compressed output files.</para> + + <para>Unless the <option>-h</option>, or <option>--help</option> option is given, + one of the commands below must be present.</para> + + <variablelist> + <varlistentry><term><option>packages</option></term> + <listitem><para> + The packages command generates a package file from a directory tree. It + takes the given directory and recursively searches it for .deb files, + emitting a package record to stdout for each. This command is + approximately equivalent to &dpkg-scanpackages;.</para> + + <para>The option <option>--db</option> can be used to specify a binary caching DB.</para></listitem> + </varlistentry> + + <varlistentry><term><option>sources</option></term> + <listitem><para> + The <literal>sources</literal> command generates a source index file from a directory tree. + It takes the given directory and recursively searches it for .dsc files, + emitting a source record to stdout for each. This command is approximately + equivalent to &dpkg-scansources;.</para> + <para> + If an override file is specified then a source override file will be + looked for with an extension of .src. The --source-override option can be + used to change the source override file that will be used.</para></listitem> + </varlistentry> + + <varlistentry><term><option>contents</option></term> + <listitem><para> + The <literal>contents</literal> command generates a contents file from a directory tree. It + takes the given directory and recursively searches it for .deb files, + and reads the file list from each file. It then sorts and writes to stdout + the list of files matched to packages. Directories are not written to + the output. If multiple packages own the same file then each package is + separated by a comma in the output.</para> + <para> + The option <option>--db</option> can be used to specify a binary caching DB.</para></listitem> + </varlistentry> + + <varlistentry><term><option>release</option></term> + <listitem><para> + The <literal>release</literal> command generates a Release file from a + directory tree. It recursively searches the given directory for + uncompressed and compressed <filename>Packages</filename>, + <filename>Sources</filename>, <filename>Contents</filename>, + <filename>Components</filename> and <filename>icons</filename> files as + well as <filename>Release</filename>, <filename>Index</filename> and + <filename>md5sum.txt</filename> files by default + (<literal>APT::FTPArchive::Release::Default-Patterns</literal>). + Additional filename patterns can be added by listing them in + <literal>APT::FTPArchive::Release::Patterns</literal>. It then writes to + stdout a <filename>Release</filename> file containing (by default) an MD5, + SHA1, SHA256 and SHA512 digest for each file.</para> + <para> + Values for the additional metadata fields in the Release file are + taken from the corresponding variables under + <literal>APT::FTPArchive::Release</literal>, + e.g. <literal>APT::FTPArchive::Release::Origin</literal>. The supported fields + are <literal>Origin</literal>, <literal>Label</literal>, <literal>Suite</literal>, + <literal>Version</literal>, <literal>Codename</literal>, <literal>Date</literal>, + <literal>NotAutomatic</literal>, <literal>ButAutomaticUpgrades</literal>, + <literal>Acquire-By-Hash</literal>, <literal>Valid-Until</literal>, + <literal>Signed-By</literal>, <literal>Architectures</literal>, + <literal>Components</literal> and <literal>Description</literal>.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>generate</option></term> + <listitem><para> + The <literal>generate</literal> command is designed to be runnable from a cron script and + builds indexes according to the given config file. The config language + provides a flexible means of specifying which index files are built from + which directories, as well as providing a simple means of maintaining the + required settings.</para></listitem> + </varlistentry> + + <varlistentry><term><option>clean</option></term> + <listitem><para> + The <literal>clean</literal> command tidies the databases used by the given + configuration file by removing any records that are no longer necessary.</para></listitem> + </varlistentry> + </variablelist> + </refsect1> + + <refsect1><title>The Generate Configuration</title> + <para> + The <literal>generate</literal> command uses a configuration file to describe the + archives that are going to be generated. It follows the typical ISC + configuration format as seen in ISC tools like bind 8 and dhcpd. + &apt-conf; contains a description of the syntax. Note that the generate + configuration is parsed in sectional manner, but &apt-conf; is parsed in a + tree manner. This only effects how the scope tag is handled.</para> + + <para> + The generate configuration has four separate sections, each described below.</para> + + <refsect2><title><literal>Dir</literal> Section</title> + <para> + The <literal>Dir</literal> section defines the standard directories needed to + locate the files required during the generation process. These + directories are prepended certain relative paths defined in later + sections to produce a complete an absolute path.</para> + <variablelist> + <varlistentry><term><option>ArchiveDir</option></term> + <listitem><para> + Specifies the root of the FTP archive, in a standard + Debian configuration this is the directory that contains the + <filename>ls-LR</filename> and dist nodes.</para></listitem> + </varlistentry> + + <varlistentry><term><option>OverrideDir</option></term> + <listitem><para> + Specifies the location of the override files.</para></listitem> + </varlistentry> + + <varlistentry><term><option>CacheDir</option></term> + <listitem><para> + Specifies the location of the cache files.</para></listitem> + </varlistentry> + + <varlistentry><term><option>FileListDir</option></term> + <listitem><para> + Specifies the location of the file list files, + if the <literal>FileList</literal> setting is used below.</para></listitem> + </varlistentry> + </variablelist> + </refsect2> + + <refsect2><title><literal>Default</literal> Section</title> + <para> + The <literal>Default</literal> section specifies default values, and settings + that control the operation of the generator. Other sections may override + these defaults with a per-section setting.</para> + <variablelist> + <varlistentry><term><option>Packages::Compress</option></term> + <listitem><para> + Sets the default compression schemes to use + for the package index files. It is a string that contains a space + separated list of at least one of the compressors configured via the + <option>APT::Compressor</option> configuration scope. + The default for all compression schemes is '. gzip'.</para></listitem> + </varlistentry> + + <varlistentry><term><option>Packages::Extensions</option></term> + <listitem><para> + Sets the default list of file extensions that are package files. + This defaults to '.deb'.</para></listitem> + </varlistentry> + + <varlistentry><term><option>Sources::Compress</option></term> + <listitem><para> + This is similar to <literal>Packages::Compress</literal> + except that it controls the compression for the Sources files.</para></listitem> + </varlistentry> + + <varlistentry><term><option>Sources::Extensions</option></term> + <listitem><para> + Sets the default list of file extensions that are source files. + This defaults to '.dsc'.</para></listitem> + </varlistentry> + + <varlistentry><term><option>Contents::Compress</option></term> + <listitem><para> + This is similar to <literal>Packages::Compress</literal> + except that it controls the compression for the Contents files.</para></listitem> + </varlistentry> + + <varlistentry><term><option>Translation::Compress</option></term> + <listitem><para> + This is similar to <literal>Packages::Compress</literal> + except that it controls the compression for the Translation-en master file.</para></listitem> + </varlistentry> + + <varlistentry><term><option>DeLinkLimit</option></term> + <listitem><para> + Specifies the number of kilobytes to delink (and + replace with hard links) per run. This is used in conjunction with the + per-section <literal>External-Links</literal> setting.</para></listitem> + </varlistentry> + + <varlistentry><term><option>FileMode</option></term> + <listitem><para> + Specifies the mode of all created index files. It + defaults to 0644. All index files are set to this mode with no regard + to the umask.</para></listitem> + </varlistentry> + + <varlistentry><term><option>LongDescription</option></term> + <listitem><para> + Specifies whether long descriptions should be included in the <filename>Packages</filename> file or split + out into a master <filename>Translation-en</filename> file.</para></listitem> + </varlistentry> + </variablelist> + </refsect2> + + <refsect2><title><literal>TreeDefault</literal> Section</title> + <para> + Sets defaults specific to <literal>Tree</literal> sections. All of these + variables are substitution variables and have the strings $(DIST), + $(SECTION) and $(ARCH) replaced with their respective values.</para> + + <variablelist> + <varlistentry><term><option>MaxContentsChange</option></term> + <listitem><para> + Sets the number of kilobytes of contents + files that are generated each day. The contents files are round-robined + so that over several days they will all be rebuilt.</para></listitem> + </varlistentry> + + <varlistentry><term><option>ContentsAge</option></term> + <listitem><para> + Controls the number of days a contents file is allowed + to be checked without changing. If this limit is passed the mtime of the + contents file is updated. This case can occur if the package file is + changed in such a way that does not result in a new contents file + [override edit for instance]. A hold off is allowed in hopes that new + .debs will be installed, requiring a new file anyhow. The default is 10, + the units are in days.</para></listitem> + </varlistentry> + + <varlistentry><term><option>Directory</option></term> + <listitem><para> + Sets the top of the .deb directory tree. Defaults to + <filename>$(DIST)/$(SECTION)/binary-$(ARCH)/</filename></para></listitem> + </varlistentry> + + <varlistentry><term><option>SrcDirectory</option></term> + <listitem><para> + Sets the top of the source package directory tree. Defaults to + <filename>$(DIST)/$(SECTION)/source/</filename></para></listitem> + </varlistentry> + + <varlistentry><term><option>Packages</option></term> + <listitem><para> + Sets the output Packages file. Defaults to + <filename>$(DIST)/$(SECTION)/binary-$(ARCH)/Packages</filename></para></listitem> + </varlistentry> + + <varlistentry><term><option>Sources</option></term> + <listitem><para> + Sets the output Sources file. Defaults to + <filename>$(DIST)/$(SECTION)/source/Sources</filename></para></listitem> + </varlistentry> + + <varlistentry><term><option>Translation</option></term> + <listitem><para> + Sets the output Translation-en master file with the long descriptions if they + should be not included in the Packages file. Defaults to + <filename>$(DIST)/$(SECTION)/i18n/Translation-en</filename></para></listitem> + </varlistentry> + + <varlistentry><term><option>InternalPrefix</option></term> + <listitem><para> + Sets the path prefix that causes a symlink to be + considered an internal link instead of an external link. Defaults to + <filename>$(DIST)/$(SECTION)/</filename></para></listitem> + </varlistentry> + + <varlistentry><term><option>Contents</option></term> + <listitem><para> + Sets the output Contents file. Defaults to + <filename>$(DIST)/$(SECTION)/Contents-$(ARCH)</filename>. If this setting causes multiple + Packages files to map onto a single Contents file (as is the default) + then <command>apt-ftparchive</command> will integrate those package files + together automatically.</para></listitem> + </varlistentry> + + <varlistentry><term><option>Contents::Header</option></term> + <listitem><para> + Sets header file to prepend to the contents output.</para></listitem> + </varlistentry> + + <varlistentry><term><option>BinCacheDB</option></term> + <listitem><para> + Sets the binary cache database to use for this + section. Multiple sections can share the same database.</para></listitem> + </varlistentry> + + <varlistentry><term><option>FileList</option></term> + <listitem><para> + Specifies that instead of walking the directory tree, + <command>apt-ftparchive</command> should read the list of files from the given + file. Relative files names are prefixed with the archive directory.</para></listitem> + </varlistentry> + + <varlistentry><term><option>SourceFileList</option></term> + <listitem><para> + Specifies that instead of walking the directory tree, + <command>apt-ftparchive</command> should read the list of files from the given + file. Relative files names are prefixed with the archive directory. + This is used when processing source indexes.</para></listitem> + </varlistentry> + </variablelist> + </refsect2> + + <refsect2><title><literal>Tree</literal> Section</title> + <para> + The <literal>Tree</literal> section defines a standard Debian file tree which + consists of a base directory, then multiple sections in that base + directory and finally multiple Architectures in each section. The exact + pathing used is defined by the <literal>Directory</literal> substitution variable.</para> + <para> + The <literal>Tree</literal> section takes a scope tag which sets the + <literal>$(DIST)</literal> variable and defines the root of the tree + (the path is prefixed by <literal>ArchiveDir</literal>). + Typically this is a setting such as <filename>dists/&debian-stable-codename;</filename>.</para> + <para> + All of the settings defined in the <literal>TreeDefault</literal> section can be + used in a <literal>Tree</literal> section as well as three new variables.</para> + <para> + When processing a <literal>Tree</literal> section <command>apt-ftparchive</command> + performs an operation similar to: + <programlisting> +for i in Sections do + for j in Architectures do + Generate for DIST=scope SECTION=i ARCH=j + </programlisting></para> + + <variablelist> + <varlistentry><term><option>Sections</option></term> + <listitem><para> + This is a space separated list of sections which appear + under the distribution; typically this is something like + <literal>main contrib non-free</literal></para></listitem> + </varlistentry> + + <varlistentry><term><option>Architectures</option></term> + <listitem><para> + This is a space separated list of all the architectures that appear under + search section. The special architecture 'source' is used to indicate + that this tree has a source archive. The architecture 'all' signals that + architecture specific files like <filename>Packages</filename> should not + include information about architecture <literal>all</literal> packages in + all files as they will be available in a dedicated file. + </para></listitem> + </varlistentry> + + <varlistentry><term><option>LongDescription</option></term> + <listitem><para> + Specifies whether long descriptions should be included in the <filename>Packages</filename> file or split + out into a master <filename>Translation-en</filename> file.</para></listitem> + </varlistentry> + + <varlistentry><term><option>BinOverride</option></term> + <listitem><para> + Sets the binary override file. The override file + contains section, priority and maintainer address information.</para></listitem> + </varlistentry> + + <varlistentry><term><option>SrcOverride</option></term> + <listitem><para> + Sets the source override file. The override file + contains section information.</para></listitem> + </varlistentry> + + <varlistentry><term><option>ExtraOverride</option></term> + <listitem><para> + Sets the binary extra override file.</para></listitem> + </varlistentry> + + <varlistentry><term><option>SrcExtraOverride</option></term> + <listitem><para> + Sets the source extra override file.</para></listitem> + </varlistentry> + </variablelist> + </refsect2> + + <refsect2><title><literal>BinDirectory</literal> Section</title> + <para> + The <literal>bindirectory</literal> section defines a binary directory tree + with no special structure. The scope tag specifies the location of + the binary directory and the settings are similar to the <literal>Tree</literal> + section with no substitution variables or + <literal>Section</literal><literal>Architecture</literal> settings.</para> + <variablelist> + <varlistentry><term><option>Packages</option></term> + <listitem><para> + Sets the Packages file output.</para></listitem> + </varlistentry> + + <varlistentry><term><option>Sources</option></term> + <listitem><para> + Sets the Sources file output. At least one of + <literal>Packages</literal> or <literal>Sources</literal> is required.</para></listitem> + </varlistentry> + + <varlistentry><term><option>Contents</option></term> + <listitem><para> + Sets the Contents file output (optional).</para></listitem> + </varlistentry> + + <varlistentry><term><option>BinOverride</option></term> + <listitem><para> + Sets the binary override file.</para></listitem> + </varlistentry> + + <varlistentry><term><option>SrcOverride</option></term> + <listitem><para> + Sets the source override file.</para></listitem> + </varlistentry> + + <varlistentry><term><option>ExtraOverride</option></term> + <listitem><para> + Sets the binary extra override file.</para></listitem> + </varlistentry> + + <varlistentry><term><option>SrcExtraOverride</option></term> + <listitem><para> + Sets the source extra override file.</para></listitem> + </varlistentry> + + <varlistentry><term><option>BinCacheDB</option></term> + <listitem><para> + Sets the cache DB.</para></listitem> + </varlistentry> + + <varlistentry><term><option>PathPrefix</option></term> + <listitem><para> + Appends a path to all the output paths.</para></listitem> + </varlistentry> + + <varlistentry><term><option>FileList</option></term><term><option>SourceFileList</option></term> + <listitem><para> + Specifies the file list file.</para></listitem> + </varlistentry> + </variablelist> + </refsect2> + </refsect1> + + + <refsect1><title>The Binary Override File</title> + <para>The binary override file is fully compatible with &dpkg-scanpackages;. It + contains four fields separated by spaces. The first field is the package name, + the second is the priority to force that package to, the third is + the section to force that package to and the final field is the maintainer + permutation field.</para> + <para>The general form of the maintainer field is: + <literallayout>old [// oldn]* => new</literallayout> + or simply, + <literallayout>new</literallayout> + The first form allows a double-slash separated list of old email addresses + to be specified. If any of those are found then new is substituted for the + maintainer field. The second form unconditionally substitutes the + maintainer field.</para> + </refsect1> + + + <refsect1><title>The Source Override File</title> + <para> + The source override file is fully compatible with &dpkg-scansources;. It + contains two fields separated by spaces. The first field is the source + package name, the second is the section to assign it.</para> + </refsect1> + + <refsect1><title>The Extra Override File</title> + <para> + The extra override file allows any arbitrary tag to be added or replaced + in the output. It has three columns, the first is the package, the second is + the tag and the remainder of the line is the new value.</para> + </refsect1> + + <refsect1><title>options</title> + &apt-cmdblurb; + + <variablelist> + <varlistentry> + <term><option>--md5</option></term> + <term><option>--sha1</option></term> + <term><option>--sha256</option></term> + <term><option>--sha512</option></term> + <listitem><para> + Generate the given checksum. These options default to on, when turned off the generated + index files will not have the checksum fields where possible. + Configuration Items: <literal>APT::FTPArchive::<replaceable>Checksum</replaceable></literal> and + <literal>APT::FTPArchive::<replaceable>Index</replaceable>::<replaceable>Checksum</replaceable></literal> where + <literal><replaceable>Index</replaceable></literal> can be <literal>Packages</literal>, <literal>Sources</literal> or + <literal>Release</literal> and <literal><replaceable>Checksum</replaceable></literal> can be <literal>MD5</literal>, + <literal>SHA1</literal>, <literal>SHA256</literal> or <literal>SHA512</literal>.</para></listitem> + </varlistentry> + + <varlistentry><term><option>-d</option></term><term><option>--db</option></term> + <listitem><para> + Use a binary caching DB. This has no effect on the generate command. + Configuration Item: <literal>APT::FTPArchive::DB</literal>.</para></listitem> + </varlistentry> + + <varlistentry><term><option>-q</option></term><term><option>--quiet</option></term> + <listitem><para> + Quiet; produces output suitable for logging, omitting progress indicators. + More q's will produce more quiet up to a maximum of 2. You can also use + <option>-q=#</option> to set the quiet level, overriding the configuration file. + Configuration Item: <literal>quiet</literal>.</para></listitem> + </varlistentry> + + <varlistentry><term><option>--delink</option></term> + <listitem><para> + Perform Delinking. If the <literal>External-Links</literal> setting is used then + this option actually enables delinking of the files. It defaults to on and + can be turned off with <option>--no-delink</option>. + Configuration Item: <literal>APT::FTPArchive::DeLinkAct</literal>.</para></listitem> + </varlistentry> + + <varlistentry><term><option>--contents</option></term> + <listitem><para> + Perform contents generation. When this option is set and package indexes + are being generated with a cache DB then the file listing will also be + extracted and stored in the DB for later use. When using the generate + command this option also allows the creation of any Contents files. The + default is on. + Configuration Item: <literal>APT::FTPArchive::Contents</literal>.</para></listitem> + </varlistentry> + + <varlistentry><term><option>-s</option></term><term><option>--source-override</option></term> + <listitem><para> + Select the source override file to use with the <literal>sources</literal> command. + Configuration Item: <literal>APT::FTPArchive::SourceOverride</literal>.</para></listitem> + </varlistentry> + + <varlistentry><term><option>--readonly</option></term> + <listitem><para> + Make the caching databases read only. + Configuration Item: <literal>APT::FTPArchive::ReadOnlyDB</literal>.</para></listitem> + </varlistentry> + + <varlistentry><term><option>-a</option></term><term><option>--arch</option></term> + <listitem><para>Accept in the <literal>packages</literal> and <literal>contents</literal> + commands only package files matching <literal>*_arch.deb</literal> or + <literal>*_all.deb</literal> instead of all package files in the given path. + Configuration Item: <literal>APT::FTPArchive::Architecture</literal>.</para></listitem> + </varlistentry> + + <varlistentry><term><option>APT::FTPArchive::AlwaysStat</option></term> + <listitem><para> + &apt-ftparchive; caches as much as possible of metadata in a cachedb. If packages + are recompiled and/or republished with the same version again, this will lead to problems + as the now outdated cached metadata like size and checksums will be used. With this option + enabled this will no longer happen as it will be checked if the file was changed. + Note that this option is set to "<literal>false</literal>" by default as it is not recommend + to upload multiply versions/builds of a package with the same versionnumber, so in theory + nobody will have these problems and therefore all these extra checks are useless. + </para></listitem> + </varlistentry> + + <varlistentry><term><option>APT::FTPArchive::LongDescription</option></term> + <listitem><para> + This configuration option defaults to "<literal>true</literal>" and should only be set to + <literal>"false"</literal> if the Archive generated with &apt-ftparchive; also provides + <filename>Translation</filename> files. Note that the <filename>Translation-en</filename> + master file can only be created in the generate command. + </para></listitem> + </varlistentry> + + &apt-commonoptions; + + </variablelist> + </refsect1> + +<refsect1><title>Examples</title> + +<para>To create a compressed Packages file for a directory containing +binary packages (.deb): + +<programlisting> +<command>apt-ftparchive</command> packages <replaceable>directory</replaceable> | <command>gzip</command> > <filename>Packages.gz</filename> +</programlisting></para> + +</refsect1> + + <refsect1><title>See Also</title> + <para>&apt-conf;</para> + </refsect1> + + <refsect1><title>Diagnostics</title> + <para><command>apt-ftparchive</command> returns zero on normal operation, decimal 100 on error.</para> + </refsect1> + + &manbugs; + +</refentry> |