Re: engineering an opposable thumb for guile?

[Bernard URBAN <Bernard dot Urban at meteo dot fr>]

>>>>> "Daniel" == Daniel Ortmann <ortmann@vnet.ibm.com> writes:

    Daniel> Sirs, I think the following missing guile feature is SO
    Daniel> IMPORTANT that its lack should be considered a bug of the
    Daniel> highest order.  Portability is good, but practical
    Daniel> engineering use requires that the following "opposable
    Daniel> thumb" be added to guile.  Its lack certainly stops my
    Daniel> guile usage in its tracks.  :-(

    Daniel> Thank you so much for your consideration!

    Daniel> SUMMARY:

    Daniel> I believe Guile needs flexible versions of perl's "pack"
    Daniel> and "unpack" functions.  Built in the spirit of Scheme, of
    Daniel> course.

    Daniel> CONTEXT:

    Daniel> My work is in the messy world of practical
    Daniel> seat-of-the-pants electrical engineering.  Often I cannot
    Daniel> adapt problems to myself but must adapt myself to those
    Daniel> problems.

    Daniel> The current problem, practically solvable only with perl,
    Daniel> consists of reading and writing 4-byte or 8-byte binary
    Daniel> floating point numbers from a file.  Here is some perl
    Daniel> code which does this: perl -e 'undef $/; $f = <>; print
    Daniel> join( "\n", unpack "f*", $f ), "\n"'
    Daniel> file-of-4-byte-binary-numbers

    Daniel> Guile appears to be missing a similar type of "opposable
    Daniel> thumb".  :-(

You can build a module for that very easily. 
I routinely decode in pure Guile binary files in the so called BUFR
format (this is an international portable binary format for
meteorological observations).

This format uses 24-bit integers in MSB first order
from a stream of bytes (i.e. char).
How to build such an integer from 3 successives bytes:

(define (3chars->integer c1 c2 c3)
  (+ (* (integer-expt 2 16) 
        (char->integer c1))
     (* (integer-expt 2 8) 
        (char->integer c2))
     (char->integer c3)))

How to build a inexact number from a string m, where the mantissa is
given between bits 'start' and 'end' in m (MSB first order), 
and the exponent is ech. Sign is handled in the BUFR format 
by coding in the file a difference from a 'ref' reference value.

(define (decode-number vec ech ref)
  (display vec) (newline)
  (let ((c (map (lambda (x y) (* x (integer-expt 2 y))) 
                (reverse (vector->list vec))
                (iota (vector-length vec)))))
    (/ (apply + ref c) (expt 10 ech))))

(define (get-bits m start end ech ref)
  (let ((vec (make-vector (- end start))))
    (do ((i start (+ i 1)))
        ((= i end) #t)
      (vector-set! vec (- i start) 
                   (bit-extract (char->integer (string-ref m (quotient i 8))) 
                                (- 7 (remainder i 8))
                                (- 8 (remainder i 8)))))
    (decode-number vec ech ref)))

(You can improve the above a lot if memory allocation is an issue)

    Daniel> PROPOSED SOLUTION:

    Daniel> Scheme should be able to generate clean and efficient
    Daniel> code.  I don't believe that this is only possible in C and
    Daniel> perl.

Agreed.

    Daniel> - Should support general "endianness".  - Should support
    Daniel> *all* IEEE floats that the hardware platform supports
    Daniel> (4-byte, 8-byte, extended, etc).  - Should support at
    Daniel> least the perl-like functionality marked below ...  -
    Daniel> ... but should do it with a clean Scheme design.  - Should
    Daniel> support *all* hardware integer values.  But saying "long"
    Daniel> is not enough because one machine's "long" is 4 bytes
    Daniel> while another is 8 bytes.  A lot of issues need to be
    Daniel> considered.  And don't forget "long long".  - Don't forget

You want something like 'read-long', 'read-short', etc...
Ok if you are reading only files written on the same machine as your
Guile, and with a program using the same C compiler used for compiling
Guile. But this is too restrictive. You should better use something
like 'read-uint32', 'write-int64', 'read-IEEE-float' etc..., with
perhaps some optional argument giving the endianness.
Notice that this is in the spirit of the X11 protocol, which is very
portable.

    Daniel> the sizes of various pointers, including function
    Daniel> pointers.  - Compiler alignment issues need to be
    Daniel> considered because of padding.  This is the "padding you

Here, you want a foreign function interface and be able to 
read/write C-structures... 

    Daniel> get when you specify XYZ option."  - Explicit padding
    Daniel> (mentioned below with the "x" template format character.
    Daniel> - Perhaps should issue "warning: machine dependent code!"
    Daniel> messages when used.

[...]

Regards.

-- 

B. Urban