Re: Making GC aware of overall allocation?

[Lars Arvestad <arve at nada dot kth dot se>]

Michael Livshin writes:
ML> the primitive mechanism in Guile is the function `scm_done_malloc'.
ML> it takes a number, which is supposed to be the malloced size.  it
ML> can be negative, so that you can say to the GC how much memory you
ML> just freed.

Thanks for this suggestion, but it is only partial solution for
me. For one of my data structures I don't have a closed form
expression for the necessary memory. I basically have to mimic the
allocation algorithm for this datastructure.

ML> it's probably a good idea to provide some sort of malloc/free wrappers
ML> with this bookkeeping, if you can put up with wasted space, something
ML> like this (written on the spot, untested)

ML> /* maximum alignment on your system, the below is good for x86 */
ML> #define SCM_MAX_ALIGN 4

I don't really mind small amounts of wasted space, but I don't
understand what the SCM_MAX_ALIGN is necessary for. To protect against 
size = 0 problems?

>> Miroslav Silovic writes:
MS> Use scm_must_malloc instead of malloc in your C code. Make sure that
MS> the free function for your smobs returns the accurate value for the
MS> memory freed (i.e. matching the ammount allocated with
MS> scm_must_malloc), otherwise your gc will get very confused.
>> 
>> But this seems to be the wrong approach IMHO. If I get my hands on a
>> library that I want to access from Guile, I should not have to rewrite 
>> important parts of it.

ML> well, in that case you want a plug-compatible malloc replacement to
ML> do the bookkeeping.

ML> I don't think you can have smart things happenning in the allocator
ML> without touching the allocator.

ML> or do I misunderstand the problem?

A plug-compatible malloc would be great, but how do I make that
happen? Should I try to link my lib with the malloc replacement before 
linking it with guile? I guess I could take a look at Electric Fence
(great debug tool, btw) for example code.

My wishful thinking has been that the GC could look at the heap size
once in a while (waving my hands here). If the heap was large and/or had
grown more than what could be explained from Guile-side growth, a GC
could be tried in order to release memory (yet more waving). I don't
know Unix and Guile well enough to say wether this is possible or not.

>> In my case, I want to eventually distribute a C library to be linked
>> in any application and do not want any dependencies on Guile. I could provide my own version of
>> scm_must_malloc which would essentially be an alias for malloc(), but
>> this will in the end evolve bugs with other programmers not honoring
>> the Guile legacy for instance.

ML> sorry, I don't understand this paragraph ;(

What I tried to say was that (1) I don't want my lib to be Guile
dependent, and (2) that if I use Guile functions in my library code
anyway, I could provide replacements (for e.g. scm_must_malloc) but
that is a bad idea because the non-Guile programmers (and myself by
mistake) will use the good old malloc, thus introducing bugs. My point
is basically that a general library should be allowed to be unaware of 
libguile. 

Sorry for bringing you into that state! I do appreciate your and
Miroslav's advice.

	Thanks,
	Lars

-- 
Lars Arvestad               Dept. of Numerical Analysis and Computing Science
                       Royal Institute of Technology (KTH), Stockholm, Sweden