Re: ppc64 vDSO in mainline

[Benjamin Herrenschmidt <benh at kernel dot crashing dot org>]

On Wed, 2005-03-23 at 17:13 -0600, Steve Munroe wrote:

> I am starting to look at conecting glibc to to the VDSO 
> __kernel_gettimeofday(). We have a few things to work out.
> 
> So far my attempts to use (almost) normal require that the application be 
> linked to vdso*.so to use extern or even dlsym resolution. 
> 
> extern  int  __attribute__ ((weak)) __kernel_gettimeofday(struct timeval 
> *, void *);
> 
> It seem that ld will not put the reference into the .dynsym table unless 
> it has found the symbol in a dso at link time. Link to the vdso*.so also 
> allow the extern/dlsym to resolve.

Can't libc.so itself link against the vDSO rather than the application ?
I don't like the idea of requiring _applications_ to link against it.

 .../...

> So this becomes the first decision point. For binding libc functions 
> internally to vdso function we could do ld.so magic to get the entry 
> points directly out of the ELF HDR or out of the link_map (BTW I would 
> appreciate some pointers in that direction as Alan Modra is on vacation 
> ... not which function to call & parms to use.) But that is not a general 
> solution and is unsatisfactory for performance sensitive functions like 
> memcpy/memcmp (because it implies another level of indirection)

Yes. Also beware that the vDSO doesn't expose function descriptors, but
the symbols exposed are offsets from the beginning of the vDSO. You'll
need your own 'branch island', if possible in asm. One advantage though
is that the vDSO does use nor affects the TOC, thus a call to a vDSO
function doesn't need a TOC load nor a TOC reload. That can probably be
hanlded with appropriate inline asm glue

> The first thing we would like is vdso call via PLT. This requires that the 
> VDSO exports enough info to resolve the PLT ref. This seems simple for 
> ppc32 as the target function address is needed from the link map. For 
> PPC64 (and other 64-bit platforms) the VDSO will have the export 
> preresolved .opd entries (The .opd entry is simply copied to the PLT 
> entry).

The vDSO will not expose anything that isn't completely static. The vDSO
is mapped read-only and shared between every processes, but can be
mapped at different virtual addresses (the later isn't yet implemented,
but the ability to do it is a design requirement). Thus it can't
expose .opd entries that have already been resolved to point to the
right virtual address as those can be different for every process. This
is why the vDSO has been changed from the early implementations to not
expose function descriptors, but simple symbols that are offsets from
the start of the vDSO. This was debated many times on this list already.

(The initial implementation did provide OPDs, and ld.so could remap them
using mprotect to trigger COW capability of the ppc64 vDSO, but that
required a change to ld.so to enable relocation on vDSOs that was
rejected, besides, I agree that it's not the right thing to do for
performances reasons).

> Ben:
> 
> I did manage to call __kernel_gettimeofday(), but
> 
> There is also a problem with the current ppc vdso64.so where it exports 
> the actual function address instead of the expected function descriptor 
> address (.opd entry). 

Not even the address, unless I did something wrong, but an offset.

> This requires the caller to manufacture a function 
> descriptor at runtime and disallows any thoughts of normal (PLT) call 
> binding. It would be helpful if the vdso64.so followed the ABI but the 
> kernel will have to insure that the .opd entries are pre-resolved.

See debates on this list monthes ago :) I did that upon request from
Ulrich, Roland, ...

> There is also the question of the TOC. The currently all functions are 
> leaf routines so a TOC is not technically required. But normal binding 
> will will (try to) load a new TOC for the VDSO anyway.

vDSO will _not_ and never use/require a TOC by design. The compiler of
course doesn't know it, but if the symbol is to be called by glibc
itself, an appropriate asm glue can be used that avoids these. If the
vDSO symbol is to be directly "linked" to the application by ld.so, then
OPD's have to be constructed on the fly by ld.so, the value of the TOC
in them is irrelevant.

Note that Paulus and I have been wondering wether it would be worth
defining a new relocation type for the vDSO symbols to avoid special
casing in ld.so. That relocation type would be for use by the vDSO,
defined as "offset to function with no TOC", and might make things
easier for ld.so to deal with those ... 

Ben.