Re: Making Guile slower (3.4 and 6.7%). (was Re: NIMP thing)

[Greg Harvey <Greg dot Harvey at thezone dot net>]

Mikael Djurfeldt <mdj@mdj.nada.kth.se> writes:

> Greg Harvey <Greg.Harvey@thezone.net> writes:
> 
> > Mikael Djurfeldt <mdj@mdj.nada.kth.se> writes:
> > 
> > > In any case, I configured a Guile from 1999-12-10 and a current Guile
> > > with configure option --disable-shared, compiled, and ran the `eval1'
> > > benchmark on an i686 and a sparc machine.  The results are 3.4%
> > > slowdown on the i686 and 6.7% slowdown on the sparc.  While one should
> > > of course always be suspicious of benchmarks 
> > 
> > Particularly when the benchmark does something trivial that only
> > covers a small portion of the evaluator.
> 
> Previously I tried to convey the idea that the right kind of benchmark
> in this case is a worst-case benchmark.  (My benchmark isn't---I just
> grabbed some code from another benchmark.)

Well, a worse case could be having a ton of NIMPS all in a row as
compared to just the test, and that runs better on a k6 ;).

> _After_ we, by using such a benchmark, get information of _how_ the
> performance has been influence, we can try to understand _why_.

Yes, but we don't know exactly what, of all the things that changed,
is causing this decrease in performance; it could very well be the
NIMP change, but that benchmark doesn't tell us that, or even
necessarily suggest it (some of the drift in the pentium case is being
caused by gc, it could also be that other changes have caused a
slightly different layout of the heap, so that allocation time is a
factor, etc), which is why a good benchmark of the impact of the NIMP
changes really needs to be done with a guile that doesn't have them,
and the exact same guile with 'em, and then see if the numbers show up
(is that 10-12 before or after the allocated patch? That will slightly
slow down allocation, as well).

> When we understand why, we can better judge if this impact is
> something we want to take or not.

Certainly, but we don't know that this is the result of the NIMP, or
any number of other things.

-- 
Gregh