[PATCH,trunk+2.20] Fix issues in ld.bfd plugin interface [0/6]

[Dave Korn <dave dot korn dot cygwin at gmail dot com>]

    Hi list,

  This set of patches (following, rather than attached) aims at fixing the
problems currently extant in the first implementation of the linker plugin API
in LD.  As well as fixing the linking issues, there are a couple of related
bugfixes that arose in the course of testing.  Relevant PRs include:

http://sourceware.org/bugzilla/show_bug.cgi?id=12369
Symbols present in LTO symbol table resolved as PREVAILING_DEF_IRONLY and
optimized out appears in final symbol table.

http://sourceware.org/bugzilla/show_bug.cgi?id=12248
Linker plugin failed to preserve linking order

http://sourceware.org/bugzilla/show_bug.cgi?id=12365
undefined references produced by linker plugin are silently ignored

http://sourceware.org/bugzilla/show_bug.cgi?id=12323
linker plugin does not handle TLS

  The patches (to follow, as replies to this post) contain the following changes:

* 001ld-pe-orphan-section-alignment.diff

  Fixes a problem with orphan section alignment in the PE-COFF linker, this
was causing "ld -r" of objects containing LTO sections to break.

* 002ld-plugin-api-no-ironly-suffix.diff

  Removes the ugly name suffix from the dummy IR BFDs as previously discussed.

* 003ld-plugin-api-link-order.diff

  One of the two main functional changes to the plugin API.  This patch works
by taking all the files added by the lto plugin and inserting them into the
list of input statements at a point immediately after the first claimed object
file.

* 004ld-pr47527.diff

  The dummy IR BFDs added by the plugin API don't have any format-specific
contents, such as ELF attributes and ARM EABI versions.  This was causing
lang_check to barf on imaginary merging problems; avoided by not performing
the checks on IR files.

* 005ld-link-stage2.diff

  This is the second main functional change, and it's really the only way to
resolve the problems caused by discrepancies between the initial set of
symbols returned by the plugin based on the LTO symtabs in the IR files, and
the actual set of symbols used and defined by the eventual object file(s)
added to the link after LTRANS.  It's largely the same approach as HJ's
2-stage link, except that it closes and reopens the existing input BFDs in
place, rather than adding a second set of input statements.  This results in
some different behaviour between HJ's linker and this patched version, which I
will mention in the section below with test results.

* 006ld-plugin-sym-wrappers.diff

  I found this running the GCC testsuite, where on cygwin we use ld wrappers
to allow replacement of libstdc++ new/delete operators.

* 00xld-lto-test-tweaks.diff

  This is out of the main series, because it's a couple of tweaks to the LTO
tests in HJ's branch; included here for reference, since I've been running
tests with it folded in.  I notice it's slightly outdated here, as upstream HJ
has already added gcc_ld_flag (and gcc_gas_flag, in fact) to the ld/12365 test
in lto.exp, so that bit won't apply cleanly, but the second change in that
hunk (adding a '_?' prefix to the regex) is still needed.  FYI, YMMV.

  Results: current binutils HEAD gives the following failures from HJ's
ld-plugin/lto.exp tests, on i686-pc-cygwin:

> FAIL: PR ld/12365
> FAIL: ld-plugin/lto-4r-a
> FAIL: ld-plugin/lto-4r-b
> FAIL: ld-plugin/lto-4r-c
> FAIL: ld-plugin/lto-4r-d
> FAIL: LTO 5 symbol
> FAIL: LTO 3b
> FAIL: LTO 4a
> FAIL: LTO 4c
> FAIL: LTO 4d
> FAIL: LTO 5
> FAIL: LTO 10

  HJ's mixed-lto branch fixes 12365 and handles mixed IR and non-IR object files:

> FAIL: ld-plugin/lto-10r
> FAIL: LTO 5 symbol
> FAIL: LTO 3b
> FAIL: LTO 4a
> FAIL: LTO 4c
> FAIL: LTO 4d
> FAIL: LTO 5
> FAIL: LTO 10

  After my patches, LD gives the following results:

> FAIL: ld-plugin/lto-4r-a
> FAIL: ld-plugin/lto-4r-b
> FAIL: ld-plugin/lto-4r-c
> FAIL: ld-plugin/lto-4r-d
> FAIL: LTO 4a
> FAIL: LTO 4c
> FAIL: LTO 4d
> FAIL: LTO 10
> FAIL: LTO 12a
> FAIL: LTO 12b
> FAIL: LTO 15

  Relative to current LD, that has fixes to 12365, 5 symbol, 3b and 5.  The
new FAILures on 12a/b and 15 are actually a difference of interpretation about
how the linker is supposed to work between me and HJ's code.  The tests assume
that it is not possible to override builtins such as __udivdi3 or memcpy, but
it works OK with the attached patches, so I'm not sure why these tests do what
they do.  For example, test LTO 15 has this source file compiled as an object:

> #include <stdio.h>
> 
> int main(int argc, char **argv)
> {
>   printf ("PASS\n");
>   return (int) ((unsigned long long) argc / argv[0][0]);
> }

... and this one in a library archive member:

> extern void abort (void);
> unsigned long long
> __udivdi3(unsigned long long n, unsigned long long d)
> {
>   abort ();
>   return n + d;
> }

... so I would have thought that the ideal case would be for an LTO link to
mimic a non-LTO link, and pull in the definition of __udivdi3 there and abort.

  That aside, the only thing that LD still doesn't handle after these patches
is mixed object files that are created by using "ld -r" to merge multiple IR
and non-IR object files.

  Patches to follow, as separate posts, so that we can discuss each in its own
thread.  I'm going to spend a couple of days testing it on 32- and 64-bit
linux, where I can also compare it to GOLD, verifying that it fixes the bugs I
think it should fix, and doing some lto-bootstrap-and-tests of GCC for
comparison, before I go committing anything, but I think the series is ready
to review so I'm posting it now.  Wouldn't mind if anyone wanted to help run
some testing on it either! :-)

    cheers,
      DaveK