A new strategy for internals documentation

[Stan Shebs <stanshebs at earthlink dot net>]

Although GDB has had an internals manual for many years, nobody has
been especially happy with it.  It is perennially out of date,
sometimes majorly so, and it is quite incomplete.  With the passage of
time, the whole idea of an internals manual has come to seem like a
throwback to the old days, when the prospective GDB hacker's
information had to come from either printed books, or tape archives
downloaded from an FTP site.

Various ideas for how to fix have been floating around, generally with
the goal of improving the manual somehow, but nothing has developed
into action.

I propose a two-pronged strategy, first migrating the internals manual
to the wiki, and then introducing Doxygen for source code
documentation.

*** Migration of gdbint.texinfo to wiki

The first proposed step is to pull the manual into the wiki and retire
it from the sources.  gdbint.texinfo includes a variety of classes of
info, and not counting the obsolete parts, they generally fall into
these groups:

* General description of architecture and algorithms

Each of these sections will work well as an individual wiki article.

* How-tos and tutorials, such as for language and target ports

Likewise, these sections will make good wiki articles.

* Procedures and standards, such as for releases and end of year

These evolve as the release manager and other owners see fit, and the
wiki's dynamic style is a better fit for keeping them up to date.

* Cross-references to more info

Much of this is redundant with information already on the GDB website
or in the wiki.

* Official description of internal API

This material has a different character, in that it is closely tied to
the current state of the sources.  In theory, this should be getting
updated in sync with source changes, but in practice, updates are
rare.  Worse, proposals to auto-generate any of it from the (GPLed)
sources run afoul of the incompatible GFDL that governs the wiki and
the manuals.  I expect that this category will not do any better as
wiki pages, and will eventually be deleted, or made into links to the
sources or Doxygen pages.

* Migration mechanics

At the mechanical level, I suggest a mostly-automatic process in which
each node of gdbint.texinfo becomes a separate wiki page, with the
navigation links preserved as wiki links.  Subsequently wiki editors
can remove redundant or obsolete material, and the wiki change history
will reflect that process.

*** Doxygen

As a second step, we should adopt Doxygen for the sources, and use it
to generate material for a new area of the website, which will be the
detailed documentation of GDB internals.

While Doxygen is not commonly used by FSF projects, it is widespread
in the open-source world.  GNU Radio and libg++ are two well-known FSF
projects using Doxygen now.

The initial policy for Doxygen usage should be to allow but not
require it for new checkins, while at the same time developing the
infrastructure that builds online documentation.  Later policy changes
could make it required for particular cases, such as for cross-file or
"official" internal API.

*** Upsides

Having the narrative and descriptive material as a pile of wiki page
will make it easier to tinker with incrementally.  Changes won't go
through a formal patch review process, which is OK because the pages
are more informal and tutorial rather than authoritative.
(Knowledgeable GDBers should still monitor edits and fix any serious
errors that creep in.)

Conversely, Doxygen in the sources will bring additional rigor and
detail to a chronically weak aspect of GDB's code, namely, how the
different parts are supposed to work with each other.

A more subtle advantage of the dual strategy is that it allows us room
to experiment with approaches.  If the wiki proves popular, then we
could end up with a sizeable and useful "GDBedia" of articles
describing different parts of the debugger in detail.  Similarly, if
Doxygen proves popular, then we might see the Doxygen section grow to
include not just the bare documentation, but a collection of
instructions and howtos.  (And if neither prove popular, we're not
any worse off than we are now. :-) )

*** Downsides

Maybe somebody will be bummed that gdbint.texinfo is gone.

As a texinfo document, it has always been possible to produce printed
versions, info files, PDF files, etc, and we will lose that
capability.  However, while this is important for the main GDB manual,
there is no evidence that any GDB hacker uses the internals manual in
these formats as guidance when working on GDB, nor do I know of
anybody selling a printed internals manual.

The licensing requires us to be careful about migrating pieces of
text; material from gdbint.texinfo can be copied to the wiki, but not
into the sources.  This is not purely hypothetical, as for instance
the gdbint.texinfo description of i386_insert_watchpoint is more
detailed than the comments currently in the source code.  In such a
case, it is allowable to refer to the gdbint description when
expanding on the comments in the source code, but not to copy
verbatim.  However, I believe the amount of material that would need
to be rewritten in this way is small, perhaps a dozen pages or so,
requiring only a few hours of work.

The existence of two places to record information may become a dilemma
for contributors.  Strictly speaking, the dilemma exists now, but in
the absence of a internals documentation strategy, we have not
insisted on much beyond a few comments in source code.  If there is
much confusion, we may need to lay out a more precise policy about
where specific kinds of information should go.

Stan
stan@codesourcery.com