Re: OO.o / ld / -Bsymbolic patch ...

[michael meeks <michael dot meeks at novell dot com>]

Hi Jakub,

On Tue, 2005-07-05 at 17:27 +0200, Jakub Jelinek wrote:
> That begs the question, are you using prelink or not?

	Not, no - partly becuase my (older) system doesn't support it out of
the box, but partly also because I really don't like the prelinking
concept at all :-) it seems to me a nasty hack to overcome a bad design
(but most likely I've misunderstood it).

> This is on a prelinked system, so I use LD_PRELOAD=libSegFault.so
> to make prelinking useless (libSegFault.so has < 30 relocations,
> so shouldn't change the numbers much):

	Interesting.

> LD_PRELOAD=libSegFault.so LD_DEBUG=statistics oowriter
>       4630:                final number of relocations: 62756
>       4630:     final number of relocations from cache: 121745
> LD_DEBUG=statistics oowriter
>       4669:                final number of relocations: 43733
>       4669:     final number of relocations from cache: 99881

	OK - so, my binutils change does slightly better without the various
complexities, latencies, limitations etc. of pre-linking :-)

> Now, there is still significant overhead processing the relocations

	Yes - of course; and much of this is caused by not only the large
number of relocations but the very long lists of libraries we link
against etc. etc.; The large number of libraries is partially caused by
pruning lots of functionality out into components that don't have to be
loaded unless necessary [ a good design philosophy with a scalable
dynamic linker ;-]

> The most expensive relocations are those not on the "from cache:" lines,
> they need the expensive symbol lookup.

	Sure - of course, the unique symbol driven relocations are the most
interesting.

> A useful experiment would be to link a bunch of soffice.bin programs

	Well - it's a good idea :-) the problem is that a chunk of OO.o
infrastructure is designed (mozilla-style) not to allow multiple
concurrent processes to access it's data structures - pwrt.
configuration data - so, having a single binary is rather important -
also; the sheer size of the code is also a factor: ~40Mb to have all
chart,writer,calc,draw [without underlying libraries] in the same
module.

	Also - as I say, I'd prefer to add options to improve the efficiency /
performance of the dynamic linker for large & componentised C++ projects
rather than put effort into prelinking.

>  each
> with the libraries current soffice.bin library already links against plus
> libraries listed in the corresponding pagein-* file, then prelink all of
> them.  I guess startup time of say swriter.bin etc. would go significantly
> lower.

	Well - the relocation cost is under 1/2 the (warm) startup cost - so
there are clearly lots of other areas for improvement too that we're
investigating. Wrt. linking all the libraries together & pre-linking: we
dlopen many of them & introspect them to grab magic 'create_component'
type symbols - would that continue to work nicely wrt. a dlopen() on a
library that is already linked ?

	My other thought was to link as much of the common helper code as
possible into 1 monster 'liboo.so' library and ensure that this was 1st
after soffice.bin in the symbol search path - thus very substantially
shortening the average symbol lookup cost - perhaps by a factor of ~20
or so.

	Either way - all of the above even if it's possible, involves some
quite substantial & unpleasant re-working of the OO.o build; clearly
it'd be nice to exhaust other opportunities for improving the [dynamic]
linker (for others as well) before resorting to evil OO.o specific
workarounds like that ;-)

	What did you think of the patch ? ;-)

	Regards,

		Michael.

-- 
 michael.meeks@novell.com  <><, Pseudo Engineer, itinerant idiot