Re: binary-io (was Re: rfc 2045 base64 encoding/decoding module)

[Gary Houston <ghouston at arglist dot com>]

| From: sen_ml@eccosys.com
| Date: Thu, 03 Feb 2000 15:54:03 +0900

| (define (write-unsigned32/little-endian unsigned32)
|   (write-char
|    (integer->char
|     (logand unsigned16 #b00000000000000000000000011111111))
|    binary-io-output-port)
|   (write-char 
|    (integer->char
|     (/ (logand unsigned16 #b00000000000000001111111100000000) 256))
|    binary-io-output-port)
|   (write-char 
|    (integer->char
|     (/ (logand unsigned16 #b00000000111111110000000000000000) 32767))
|    binary-io-output-port)
|   (write-char 
|    (integer->char
|     (/ (logand unsigned16 #b11111111000000000000000000000000) 16777216))
|    binary-io-output-port))

I'm sorry about making such a late followup to this thread, but I can
see a few difficulties with this style of code.

It's not a good idea to use read-char and write-char to process
arbitrary bytes.  It would probably break if multibyte character
support was implemented (or alternatively, make multibyte character
set support harder to implement and less elegant: just take a look at
C.)

Unfortunately the alternatives aren't very good at present.  R5RS
doesn't provide anything.  Therefore it seems to me that the first
thing to do is investigate the datatypes for handling foreign data.

Some ideas on what could be done (not completely unlike some of Harvey
Stein's suggestions):

Do not write procedures that pack/unpack data directly from a port,
since they would also be useful as part of a foreign function interface
and probably for things like mmap.  Constructing a port has a certain
overhead and imposes a serial interface.

Define a new "byte-vector" type.  "byte" would be defined to be the
same thing as a C char, since Guile is supposed to be good at
interoperating with C.  Possibly call it something else out of respect
for Common Lisp usage of "byte".  It's true that bit-vectors already
exist, but I think byte-vectors would be more convenient.

Also introduce a "shared byte-vector".  This would generally behave
like a byte-vector but wouldn't free its memory when garbage
collected.  It would be created by specifying an address (from C
again, a char * expressed as an integer) and a length.

The idea is to allocate a byte-vector for reading data from a port but
use a shared byte-vector to access foreign data in already in memory.

One could also address the entire memory by creating a shared
byte-vector at address 0 and length equal to the address space, thus
giving Guile a "peek" and "poke" facility.  Hurrah!

It may also be useful to make port buffers visible as shared
byte-vectors.

Then write unpack/pack routines which operate on byte-vectors, including
conversion to and from Scheme chars and strings.