Writing Guile Modules

[forcer <forcer@mindless.com>]

[i tried to sum up some information about this. I hope this
helps. Feel free to use this in any documentation you put
together. Additionally, it would be nice if you could add the
links to the libraries and languages to the Ideas page]


With this document, I'm trying to put together some helpful guidelines on
how to write Guile wrappers for C libraries.
For modules, this is the usual use of C - writing a glue to use the C
library from the Scheme world.

The two worlds differ quite alot. E.g. where in the C world it's not uncommon
that an application crashes, such a behavior should not happen in the
Scheme world - an error condition should throw an error and be done
with it. The C glue should guarantee this.

Another aspect to mind is that C lacks a few features of Scheme. Where in
C an array doesn't contain it's length, and a string is terminated by a
\0 (and thus not 8-bit-clean), Scheme vectors and strings know their
length. So if a C function requires the passing of e.g. a void* pointer
and a size_t len, you can just use Scheme vectors or strings (or both).

In writing the Scheme wrappers, you have to find a way to hide the complexity
of C while still allowing a sensible, full-fledged use from Scheme, and
without doing too much in the C interface. Ideally, the glue should just
convert values - except when this would introduce the possibility of
inconsitencies by actions from the Scheme-world, or when this would enforce
C-specific problems on the Scheme-world.

A few examples?
- "Introducing the possibility of inconsitencies from the Scheme-world"
  Let's chose termcap here. It should be initialized with tgetent() before
  calling any other function. You can chose two approaches here:
  1) Initialize termcap in your module's initialization routine
  2) Keep a flag which says "not initialized" and throw an error on each
     use of a function until the flag says "initialized"
- "enforce C-specific problems on the Scheme-world"
  Using gdbm here. Gdbm works with "datums", which is basically a void pointer
  to the data, and a length for the number of bytes there.
  Enforcing the user to create a datum first which can be passed to the
  functions is short-sightened - a Guile string already has this information,
  so the C glue should just take a Guile string.

Scheme is different from C.
It's nice to have a familar Schemeish work environment. That is, if possible,
the interface should be functional (or at least, provide a copy function for
it's smob types).
In the termcap example above, you could make a smob for termcaps and try
to make each Scheme-level function take such a type (maybe optionally, and
having something like (current-termcap)), and use it for it's termcap.
I don't know wether that's possible with termcap, but it would be a nice idea,
and get's over the point - Scheme functions shouldn't rely on global
data structures if not necessary.

A few notes on the design:
Most of the time, a library has a library-specific struct, and alot of
functions which operate on that struct. The approach here is first to
implement a smob for the struct (or a port, if the struct can be used for
input/output, regardless how).
This smob should contain the struct, and a few bookkeeping values - maybe
a "valid"/"initialized"-flag as noted above, maybe a few cached values of
preconverted SCM values (useful for large substructures which's conversion
is time-consuming). You should also implement a few C-level
accessor/conversion functions for your smob.
Generally, you should wrap things you do more than once in functions or
macros. This will make your code alot more readable and easier to change.

Sometimes, you come across a library which just adds a layer onto
a port - e.g. SSL, encryption, compression - try to keep this a
layer. The function to create the port should accept another port
to write to/read from.

A note on strings:
If the new i18n code is implemented, and guile uses UTF-8 for it's strings,
they won't correspondent to the C {void*;int} pair anymore. If i got it
right, Guile will have a bytevector type by then, so that can be used.

Ok. You are eager to write the Best Guile Module out there, you want to code,
you have your editor and your keyboard ready, but ... which library? :)
On the Guile homepage <URL:http://www.gnu.ai.mit.edu/software/guile/> is
a nice Ideas list. Sadly, it doesn't list alot of libraries, and of those
it does list, it doesn't list the fitting URLs.
Here is my list. If there's a ? in the beginning, i couldn't reach the
homepage. Some aren't really libraries, but documentation of how
some things work, so one can implement an interface. See
yourself. Recheck the license before implementing a module, though :)

? FDF (http://beta1.adobe.com/ada/acrosdk/forms.html)
  Forms library
? FPS (http://www.mit.edu/people/wandy/fps/fps.html)
  Functional PostScript, a PostScript using Scheme instead of
  Forth
Aspell (http://metalab.unc.edu/kevina/aspell/)
  A spelling correction library
expat (http://www.jclark.com/xml/)
  XML parser
ffcall (http://clisp.cons.org/~haible/packages-ffcall.html)
  Foreign Function Call library - very nice!
GD (http://www.boutell.com/gd/)
  A library to create/modify PNG images
Ilib (http://www.radix.net/~cknudsen/Ilib/)
  A library to create/modify images of different formats
IMAP (ftp://ftp.cac.washington.edu/imap/)
  IMAP usage
LDAP (http://www.openldap.com/)
  LDAP client library
libpcap (ftp://ftp.ee.lbl.gov/libpcap-0.4.tar.Z)
  Packet-level sniffing library
libwww (http://www.w3.org/Library/)
  A library for the WorldWideWeb
mcrypt (ftp://argeas.cs-net.gr/pub/unix/mcrypt/)
  Encryption library
mhash (http://sasweb.de/mhash/)
  Hashing library
NIS/YP (http://www.desy.de/~sieversm/ypdoku/ypdoku/ypdoku.html)
  Accessing NIS from programs
ODBC (http://www.unixodbc.org/)
  ODBC client library
pdflib (http://www.ifconnection.de/~tm/)
  A library to work with PDF files
PGG (http://www.nessie.de/mroth/pgg/)
  A glue library to use GNU Privacy Guard
SSLeay (ftp://ftp.psy.uq.oz.au/pub/Crypto/SSL/)
  SSL from programs
UCD SNMP (http://ucd-snmp.ucdavis.edu/)
  SNMP usage library
WDDX (http://www.wddx.org/)
  Language-independent, XML-based representation of data
zlib (http://www.cdrom.com/pub/infozip/zlib/)
  Compression library

Also, you can browse through CPAN and take any module - most of them would
be nice to have.

If you're less of a C-person, you might find implementing guile readers
interesting. A Guile reader is a function which reads an expression from a
language, and returns an equivalent Scheme expression.
If you want to write such a reader, think about those languages (in addition
to the ones noted on the Ideas page)

- JavaScript
- Lua (http://www.tecgraf.puc-rio.br/lua/)
- Latte (http://www.latte.org/)
- PHP (http://www.php.net/)
- Python (http://www.python.org/)

  -forcer

-- 
((email . "forcer@mindless.com")       (www . "http://forcix.cx/")
 (irc   . "forcer@#StarWars (IRCnet)") (gpg . "/other/forcer.gpg"))