Re: parallelized 'ld'?

[Michael Meissner <bin-mail at the-meissners dot org>]

On Tue, Jul 15, 2003 at 12:34:31AM -0500, Cynbe ru Taren wrote:
> 
> Well, I was being a bit facetious in suggesting that linking is just
> additions, of course -- obviously there is a good deal of bookkeeping
> and bit-shuffling which is collectively likely to take more time than
> the actual relocation additions proper.
> 
> ('ld' being a core and mature GNU utility, I'm presuming all
> the low-hanging fruit in terms of monoprocessor performance
> tuning has long since been picked.  Am I being naive?)

I suspect there are still plenty of opportunities for improving ld's
performance, particularly when building very large apps.  Mozilla is probably
the largest app that most people on the open source side of things routinely
link (and hence might be tempted to optimize for).  When I was Cygnus/Redhat,
very few customers ever paid for fine tuning the tools compared to doing
initial ports, or adding specific optimizations.  Given your app is much larger
than most of the freeware apps, ld might need some attention to scaling.

For instance, you might look at whether ld is paging itself to death.  It might
make sense to have a link server that is a 64-bit machine (IA-64, Alpha,
AMD-64, Powerpc-64) and has several gigabytes of real memory.

Another place to look at is symbol lookup, as has been noted.  A quick glance
at ld shows it using the bfd_link_hash_* functions, which are defined in
bfd/linker.c.  These functions in turn use the generic hash functions in
bfd/hash.c, and in particular bfd_hash_table_init is used to create the hash
table entry.  Bfd_hash_table_init in turn calls bfd_hash_table_init_n with
DEFAULT_SIZE, which is 4051.  I strongly suspect a bandaid approach might be to
up this value by 1 or 2 orders of magnitude, and as somebody else mentioned, a
better approach is to change bfd_hash to use the dynamically expanding hash
tables in libiberity (libiberity/hashtab.c) that Vladimir Makarov wrote as part
of his haifa scheduler rewrite.

-- 
Michael Meissner
email: gnu@the-meissners.org
http://www.the-meissners.org