add optional support for comments in pkgTagFile

APT usually deals with perfectly formatted files generated automatically
be other programs – and as it has to parse multiple MBs of such files it
tries to be fast rather than forgiving.

This was always a problem if we reused this parser for files with a
deb822 syntax which are mostly written by hand however, like
apt_preferences or the deb822-style sources as these can include stray
newlines and more importantly comments all over the place.

As a stopgap we had pkgUserTagSection which deals at least with comments
before and after a given stanza, but comments in between weren't really
supported and now that we support parsing debian/control for e.g.
build-dep we face the full comment problem e.g. with comments inbetween
multi-line fields (like Build-Depends).

We can't easily deal with this on the pkgTagSection level as the interface
gives access to 'raw' char-pointers for performance reasons so we would
need to optionally add a buffer here on which we could remove comments
to hand out pointers into this buffer instead. The interface is quite
large already and supports writing stanzas as well, which does not
support comments at all either. So while in future it might make sense
to have a parser setup which deals with and keeps comments in this
commit we opt for the simpler solution for now: We officially declare
that pkgTagSection does not support comments and instead expect the
caller to deal with them, which in our case is pkgTagFile:

pkgTagFile is extended with an additional mode which can deal with
comments by dropping them from the buffer which will later form the
input of pkgTagSection. The actual implementation is slightly more
complex than this sentence suggests at first on one hand to have good
performance and on the other to allow jumping directly to stanzas with
offsets collected in a previous run (like our cache generation does it
for example).

1 files changed, 21 insertions, 2 deletions

diff --git a/apt-pkg/tagfile.h b/apt-pkg/tagfile.h
index 922b56f8e..2f26846ef 100644
--- a/apt-pkg/tagfile.h
+++ b/apt-pkg/tagfile.h
@@ -34,7 +34,15 @@
 
 class FileFd;
 class pkgTagSectionPrivate;
+class pkgTagFilePrivate;
 
+/** \class pkgTagSection parses a single deb822 stanza and provides various Find methods
+ * to extract the included values. It can also be used to modify and write a
+ * valid deb822 stanza optionally (re)ordering the fields inside the stanza.
+ *
+ * Beware: This class does \b NOT support (#-)comments in in- or output!
+ * If the input contains comments they have to be stripped first like pkgTagFile
+ * does with SUPPORT_COMMENTS flag set. */
 class pkgTagSection
 {
    const char *Section;
@@ -139,7 +147,10 @@ class pkgUserTagSection : public pkgTagSection
    virtual void TrimRecord(bool BeforeRecord, const char* &End) APT_OVERRIDE;
 };
 
-class pkgTagFilePrivate;
+/** \class pkgTagFile reads and prepares a deb822 formatted file for parsing
+ * via #pkgTagSection. The default mode tries to be as fast as possible and
+ * assumes perfectly valid (machine generated) files like Packages. Support
+ * for comments e.g. needs to be enabled explicitly. */
 class pkgTagFile
 {
    pkgTagFilePrivate * const d;
@@ -148,14 +159,22 @@ class pkgTagFile
    APT_HIDDEN bool Resize();
    APT_HIDDEN bool Resize(unsigned long long const newSize);
 
-   public:
+public:
 
    bool Step(pkgTagSection &Section);
    unsigned long Offset();
    bool Jump(pkgTagSection &Tag,unsigned long long Offset);
 
+   enum Flags
+   {
+      STRICT = 0,
+      SUPPORT_COMMENTS = 1 << 0,
+   };
+
+   void Init(FileFd * const F, pkgTagFile::Flags const Flags, unsigned long long Size = 32*1024);
    void Init(FileFd * const F,unsigned long long const Size = 32*1024);
 
+   pkgTagFile(FileFd * const F, pkgTagFile::Flags const Flags, unsigned long long Size = 32*1024);
    pkgTagFile(FileFd * const F,unsigned long long Size = 32*1024);
    virtual ~pkgTagFile();
 };