guile GC & smobs

[Jon Trowbridge <trow@emccta.com>]

OK, so thanks to the members of this list, I've cured my dreaded
"slowdown bug".  scm_mallocated was indeed getting underflowed, and
the garbage collector was going into spasms, because of a
misunderstanding about how the return value of smob free functions are
used.  I've since set them all to return 0, and things are working
just fine...

...except for one problem.  A typical "constructor" for one of my
smobs looks something like this:

SCM
foo2scm(Foo* f)
{
  SCM_DEFER_INTS;
  SCM smob;
  SCM_NEWCELL(smob);
  SCM_SETCDR(smob, f);
  SCM_SETCAR(smob, foo_type_tag);
  SCM_ALLOW_INTS;
  return smob;
}

SCM
make_foo(... some_data ...)
{
  ... blah blah blah ...
  return foo2scm(new Foo(... some_other_data ...));
}

While the "destructor" looks like this:

scm_sizet
free_foo(SCM x)
{
  Foo* f = SCM_TO_FOO(x);
  SCM_DEFER_INTS;
  delete f;
  SCM_ALLOW_INTS;
  return 0;
}

Now this works fine, and I get to play with my Foo objects in guile.
But the problem is that Foo objects could be really big, and guile's
GC has no way of knowing that this innocent looking Foo-smob cell is
really holding an object that is sucking up 3 or 4 megs of memory.

So the net result is that the garbage collection "runs behind".  When
run a small simulation that involves the creation of lots of different
smobs, my process jumps up to about 35M... which is about the right
size.  But if I run that simulation over and over again in a loop, my
process blows up to around 105M.  It isn't a memory leak; unless I'm
mistaken, I'm just not garbage collecting often enough, and all sorts
of huge unused objects are being left laying around on the heap.

Now I don't mind if gc takes place much more frequently, since 97% of
my processing is done in C++ code, and I don't think that it would
slow me down to any significant degree.  The optimal solution would be
if I could somehow tell guile that a lot more memory is being sucked
up than it suspects.  Then I could just define:

SCM
foo2scm(Foo* f)
{
  ... details omitted, see above ...
  scm_report_memory_usage(approx_size_of_a_Foo); // A made-up function
  return smob;
}

scm_sizet
free_foo(SCM x)
{
  ... details omitted, see above ...
  return approx_size_of_a_Foo;
}

But is there a way to "scm_report_memory_usage"?  Is it considered
proper to just manually increment scm_mallocated?  Or is there some
other way of dealing with this?  Or should I just get an extra 256M of
memory to put in my workstation and not worry about it too much?

-JT