Translating into Scheme, not byte code

[Mikael Djurfeldt <mdj at mdj dot nada dot kth dot se>]

I'm really looking forward to have a good look at your VM after I've
done some work.  What you write seems really exciting!

Just a boring policy-note:

Keisuke Nishida <kxn30@po.cwru.edu> writes:

> BTW, my VM has a %jump instruction, and the intermediate code of the
> compiler has #:label and #:goto keywords.  I think writing a compiler
> could be easier than writing a translator when we have a VM.

I think what we'll try to focus on, at least to start with, is to
translate into Scheme, and then, from there, to byte code.

The idea here is to modularize the process.  We want to treat the
following problems as independently from each other as possible:

1. Lang X --> Scheme problem

2. Scheme --> byte code problem

3. Scheme --> machine code problem

This has obvious gains:

* Efforts we put into solving problems 2 and 3 makes gains for all
  languages in 1.

* We get fewer problems, in total, to solve.

* Solutions to anyone of problems 1-3 can, in principle, be replaced
  with new versions or alternatives without the need to replace the
  other parts.

The idea to divide a compiler into a front-end and a back-end is
similar in spirit, and very powerful.  The success of GCC is a good
demonstration.

*** It is the current GNU policy to use Scheme as the intermediate
*** representation when combining different translators in Guile.

* Note that if we *would* decide that Guile translators have the
  option to translate into the byte code of your VM, we would be tied
  to it in an unfortunate way.  (Of course, anyone have the option to
  do anything with Guile, but this is not going to be supported.)

  It is unfortunate because

  1. We want the efforts of people writing translators to be
     long-lasting.  If they would translate into code for your VM,
     they would stop working the day we switch to something
     different.

  2. The code of your VM is, and should be, optimized for evaluating
     Scheme code efficiently in the framework of the current Guile
     implementation.  It is not (I think?) designed to be an optimal
     intermediate representation.

As has been said before, the reason why we haven't seen translators
yet, is not that it isn't a good idea to translate into Scheme.  We
simply haven't pooled resources into it yet.  We are, together, doing
that now.

Also, maybe people have a too static way of looking at what can be
done with regard to translation on both sides of the Scheme
representation?  We don't need to translate expression by expression,
but can do various forms of transformations.  We have wonderful tools
such as partial evaluators and cps.  For example, it's not necessary
to use tagbody forms or continuations to handle jumps.

Below the Scheme level, we can use type analysis to produce extremely
efficient code.  There's no reason why we couldn't transform the
Scheme representation into code nearly as efficient as compiled C.

What we need now is some practical experience with the problem so that
various people can come up with good solutions to common problems,
develop a nice toolset, and, probably foremost, begin to transfer
parts of the knowledge about translation which already exists in the
academic world into practical use in Guile.

Once we have that, we can start thinking about a new representation.

Then, I would bet we would be better off with something akin to Scheme
rather than byte code.  One could, for example, simplify the language
so that we always translate into a subset of Scheme, similar to the
"pre-scheme" of Scheme48.  We could also add new constructs specially
devised for aid of translation, such as forms carrying type
information over to the sub-Scheme translation.


So, please let's give translation into Scheme a chance first, and see
where that carries us.

[says Mikael in soft-maintainer, medium-categorical mode]