Re: primitive documentation conventions

[Jim Blandy <jimb@red-bean.com>]

> I would like to suggest first off that we should design and implement
> the scwm documentation extraction system with the intention of it
> being eventually incorporated in Guile, perhaps with some changes.

Exactly.  This is a problem that all Guile users will face, so if we
can solve it thoroughly, that's a big win.

I agree that we should implement the extractor in Guile.  It may be
slow at first, but speeding up Guile's I/O is on the list, and I'd
like to have some actual text-processing applications to test my
changes.


> * Output: We should, for now at least, output each of DocBook SGML,
> TexInfo, and an on-disk documentation string database.

Sounds reasonable.


> * How to get at docstrings for functions written in C: I suggest that
> we try for a system where they are not kept in core straight off.

Emacs does this.  Docstrings are included as an argument to a macro
which drops them when generating normal Emacs code.  Then there's
another function which scans the source code for these macros and
extracts the docstrings into a file, labeled with the function name.

I seem to remember long-standing bugs or FAQ's about this process; the
installation procedure wasn't quite right, and the docstring files
were always out of date with respect to the executable.  I haven't
thought about this at all, but if all Guile users are going to follow
this procedure, we should look for something foolproof.

> ** For C files, there are two alternatives that are, IMO, more or less
> equally acceptable. One is to use specially distinguished comments,
> the other is to add an extra string parameter to SCM_PROC which
> guile-snarf would ignore, but which the doc extractor would use for
> it's own purposes. Jim Blandy seemed suspcious of the concept of
> looking through the comments; however, I'm not sure it is that
> bad. It's certainly more expandable to documenting concepts and
> specially distinguished variables, and not just procedures.

I'm not so suspicious of it; I've done similar things before.
However, you can't make comments conditional on configure-generated
#defines, so your extracted docstrings may not always correspond to
the source that got compiled.  Something that operates post-CPP
wouldn't have this problem.

Note that, contrary to Knuth's claims, you can't generate decent
internals documentation by extracting stuff from the source.  Source
code and manuals are laid out differently.  This system is only going
to be providing point documentation, not a full manual.

That is, the FSF does the right thing in providing *both* docstrings
and a full Emacs Lisp manual, and having them be separate.  A manual
entry knows a lot more about its context than a docstring does, so the
best docstrings rely little on context, while the best manuals have
overview sections, explanations of common concepts, and then function
descriptions which operate in that context.


> * Markup: We may want extra markup in the docs in addition to the
> presumably automatically generated function signature. If this is ever
> supported, I suggest we use DocBook tags, as opposed to TexInfo or
> worse yet, something ad-hoc. It seems to me that DocBook->TexInfo is
> likely to be the easier translation, and if there's ever a DocBook
> system tht generates texi automatically, we won't have to worry about
> it ourselves at all.

DocBook is fine, but we'll need to specify which tags we allow in
docstrings, right?  I'm familiar with the general idea of SGML, DTD's,
style sheets, and DocBook, but I don't know the details.