Re: lazy library loading

[Andrew Senkevich <andrew dot n dot senkevich at gmail dot com>]

2013/2/19 Carlos O'Donell <carlos@systemhalted.org>:
> On 02/19/2013 05:16 AM, Andrew Senkevich wrote:
>>> Glad to hear that! We are always interested in new developers helping
>>> with the project.
>
> Please void top-posting, the email etiquette for this list is to bottom-post.
>
>> Very glad also!
>> Please correct me if I am wrong, but at this moment my vision of problems is:
>>
>> - To solve how analysis of inter-object data references will work. It
>> must help to be sure that library is suitable for lazy loading.
>
> Correct.
>
>> - To define name of library containing needed symbol in the set of
>> libraries in dependencies without their loading. We need to know which
>> exactly object we must load.
>> To solve it we propose to save this information statically in object’s
>> ELF during building of library or executable. Runtime liker must use
>> this information. Also required to have ability to mark some libraries
>> for lazy loading and some not (during building).
>
> I don't know if this is required or simply an optimization.
>
>> - To have ability to load library just on first call from that
>> library. Here we need to find out how current runtime linker resolves
>> symbols in loaded libraries with lazy binding, and find a way of
>> combining this existent functionality with addition of loading needed
>> library just before it. (May be reuse dlopen() functionality for
>> library loading in this case?)
>
> Correct. Yes, we have the ability to load a library at any point in time.
>
>> - In case of library containing constructor/destructor we need to have
>> ability execute constructor after loading and addition of destructor
>> to the destructor’s list. (But it works with dlopen(), so this
>> functionality seems already implemented.)
>
> Again that is correct, because of dlopen we have the ability to do this.
>
> You will need to:
>
> (a) Prevent the current dependency scanning system from loading any
>     libraries.
>
> (b) Enhance the resolver such that if the symbol requested is in a library
>     that hasn't been loaded that the library should be loaded before trying
>     to resolve the symbol.
>
> (c) Provide performance measurements on how much the additional checks
>     slow down the resolver.
>
> Cheers,
> Carlos.

Hello, Carlos.

At this moment I have first small result - I can skip loading 2
libraries of application and load them on demand when they needed.

It is standard Ubuntu 11.10 x86-64 application /usr/bin/eog
Libraries are libjpeg.so.62 and libexif.so.12

I have uploaded ld debug logs (with LD_DEBUG=files and removed PIDs
for more pleasant comparison) here:
http://pastebin.com/5WRyYWFd - origin, with default ld.so
and
http://pastebin.com/YVh9Jgmh - with my changed ld.so

In last log lines 29, 90, 1077, 1078, 1100 contain my added debug
messages about skip and loading libjpeg.so.62 and libexif.so.12


--
WBR,
Andrew