Re: Python transformer for Guile?

[Mikael Djurfeldt <mdj at mdj dot nada dot kth dot se>]

sen_ml@eccosys.com writes:

> the comments in pgen.c suggest that pgen is a parser generator.
> are you familiar w/ typical uses of pgen?

I don't know "pgen".  For information about lexical analyzers and
parser generators, look at the documentation for the GNU packages
`flex' and `bison'.  If you want a book, I can recommend "Compilers:
Principles, Techniques, and Tools" by Alfred V. Aho (also called the
"dragon book" because of the picture on its cover :).

> from another angle, looking through pythonrun.c, lead me to
> PyParser_ParseFile (parsetok.c), which in turn lead me to
> PyTokenizer_FromFile (tokenizer.c).  i have a feeling i may at least
> be in the ballpark here.

Yes, the tokenizer turns a stream of characters into a strem of
tokens.  The parser turn the stream of tokens into something
meaningful.  You're supposed to keep the tokenizer and make the parser
construct Scheme-Python instead of constructing the Python syntax tree.

> mdj> It might be a good idea to let it generate an intermediate
> mdj> representation (Python with Scheme syntax) and then use a
> mdj> translator, written in Scheme, to generate the Scheme code.  (The
> mdj> translator can have the shape of an evaluator (see e.g. SICP) but
> mdj> instead of computing values, it computes code.)  
> 
> i think i just barely follow your idea.  i guess an intermediate
> representation needs to be thought up.

I think it follows naturally from the Python language.  I'm only
suggesting for the tokenizer and parser to produce Python code, but
with Scheme syntax (e.g. (+ 1 2) instead of 1+2).

> i'll have to order sicp as i don't have a copy -- any other
> suggested reading?

I think the Aho book is good to have in any case.

> mdj> You'll also have to write a library with basic Python procedure
> mdj> and data types.
> 
> i'm not sure i quite follow you here -- if i understood your idea
> correctly, there are basically two large steps:
> 
>   1) python-code -> python-with-scheme-syntax
>      performed by code resulting from modifying existing python parser (c)

Yes.

>   2) python-with-scheme-syntax -> scheme
>      performed by translator code (scheme)
> 
> is it in step 2 that i will need this library?

The code produced by step 2 will use the library.  E.g., if Python
contains a dictionary type, then you need to make a goops class
representing this type and write corresponding methods to be called
from the generated code.

> > in python's case there might be licensing issues, i guess.
> 
> mdj> Python's license is compatible with GPL, so I don't think it'll be a
> mdj> problem.
> 
> i guess all that is needed is to include the copyright and permission
> notices are included.  does this count as compatible w/ gpl?

I guess the license is free enough that we can use it and put GPL on
it afterwards.  But I don't understand license issues very well, so I
hope someone else can correct me if I'm wrong.  What I can say for
sure is that we will be able to freely distribute it.