This is the mail archive of the binutils@sourceware.org mailing list for the binutils project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: Link order a pain, positional argument --start-group problematic


> I grew up with MS-DOS and the like, where link order did not matter (as far
> as I remember). In fact, I still find ld's link order counterintuitive. In
> my experience, it is a common source of unexpected link errors.

I believe that link order on MS-DOS/Windows does indeed matter, but
there are probably other differences that have kept you insulated from
the particular problems you're now having on Linux. For one thing, on
Windows, you're more likely to be using DLLs, which work differently
from static libraries, and make all their symbols visible regardless
of whether there are any outstanding references to them. In addition,
if a symbol remains unresolved, the Windows linker will go back and
search static libraries that preceded the reference, but only after
searching libraries that follow it (sort of as if the entire link line
were surrounded by --start-group and --end-group). To my mind, that's
merely substituting one set of unexpected problems for another.

At any rate, linking files from left to right, and the implications of
that on processing static libraries, are crucial to much software
design. Interposition -- the ability to provide a substitute version
of a library routine by placing it earlier on the link line -- is used
all the time, and has been a standard practice for over 50 years (long
pre-dating Unix and MS-DOS).

That said, yes, it is a common source of link errors. It's all too
common to get unresolved symbols simply because the missing symbol is
defined in a library that is placed too early on the command line.
It's also too common to end up linking to the wrong version of some
symbol that has multiple definitions. The frequency of these issues
frustrates those of familiar with linkers, because it's something that
(we feel) just shouldn't be so confusing.

> I personally find disturbing that a duplicate symbol may or may not be
> detected depending on the link order, which can be pretty hard to control in
> complex or even automatically-generated makefiles.

I'm not sure what's prompting this complaint, but it sounds like
you've encountered a situation where you accidentally had two
different routines with the same name, and didn't discover the problem
at link time because one or both of them were in libraries, and the
linker didn't believe the second one was needed. As far as the linker
is concerned, that's not a duplicate symbol -- it's simply a library
routine that isn't needed because the application already contains a
routine of that name.

If you're counting on the linker to complain about this case, then I'd
claim that you're using libraries for the wrong purpose. You're
probably using them as a packaging mechanism rather than a layering
mechanism.

> Reordering libraries in the makefile because of this is a pain, and
> sometimes there is the issue or circular dependencies.

Especially if there are circular dependencies, you're not using them
as a layering mechanism. You don't really have several libraries; you
have one library packaged across several files. You might want to
investigate using thin archives (see the ar command's 'T' option) as a
way to repackage several archive files as a single library (an
alternate approach to using --start-group and --end-group on the
command line).

> Furthermore, link order issues have multipled with the advent of C++. For
> example:
>
> https://stackoverflow.com/questions/3363398/g-linking-order-dependency-when-linking-c-code-to-c-code

I looked at this page, and I can assure you that has nothing to do
with C vs. C++. Even in C, if you put the library first on the command
line, the linker will pull in nothing from it, and the reference from
the object file will remain unresolved. If you happen to be using
dynamic libraries instead of static libraries, the link would work,
but it's still not the proper way to construct a link command.

A properly-layered application should have well-defined layers, each
of which depends only on symbols defined in that layer or in the
layers below it. The link line should always reflect that layering,
with lower layers following the higher ones.

> Using the --undefine flag is difficult with C++ mangled names, and not
> really practicable in large projects.

No, that's not a good strategy. It's simply a work-around for a
poorly-layered application (although the -u option does have some
reasonable advanced uses).

> One of the often-suggested work-arounds is to list static libraries twice,
> but I am not sure whether that would slow down the link process. Maybe
> someone here can shed some light on this aspect.

Also a hack for a poor library design (alas, even GCC does this with
its libraries). It usually suggests that what should be thought of as
a single library has been packaged as several separate files. A more
elegant solution would be to use thin archives or --start-group and
--end-group.

> Another common advice is to use --start-group / --end-group and/or
> --whole-archive . The problem is, the position of these command-line
> arguments is significant, and that is a problem with some build tools. For
> example, libtool does not support them:
>
>> Please also note that libtool currently does not support position
>> dependent linker flags such as --whole-archive, --start-group,
>> --add-needed, --as-needed, or -Bstatic.  Lifting that restriction
>> is planned, as possible, at some point in the future.
>
> libtool is part or the autotools, which could be considered dead as far as
> real development is concerned, so I do not think that a fix is in sight.

No, probably not. libtool is there to help you build highly-portable
software, so by design it's not going to support some of the more
advanced features. If you want to make your software highly portable,
I'd suggest spending the additional effort it will take to layer your
libraries properly.

Is there a reason you have to use libtool?

> As a work-around, I suggest adding alternative command-line arguments that
> are not position dependent. I guess in most situations a global flag like
> --whole-archive-all would provide a simpler, more intuitive and more
> reliable experience.

--whole-archive isn't really the same thing as --start-group and
--end-group (or a thin archive). It's really a way of saying "this
archive isn't really a library -- it's just a collection of object
files that I want to link". If you're using archives as a packaging
mechanism rather than a layering mechanism, it may be appropriate, but
keep in mind that the linker will happily pull in everything, rather
than selecting only the routines that are referenced. (You can turn on
link-time garbage collection to eliminate unreferenced routines after
that, but it's better not to pull them in at all.)

If you're hoping the linker will complain about all duplicate symbols
in your application, then --whole-archive is what you would need.
Alternatively, you could stop using archives as packages and build
your link command with all the individual object files.

While a global --whole-archive-all option might work for the archives
you build as part of the application, it would still be inappropriate
for the runtime and system libraries that form the bottom-most layers.

Another idea I've entertained in the past is to make an archive with
some different extension mean that it's a package rather than a
library, and that all of its members should be linked (as if linked
with --whole-archive). (On 32-bit HP-UX on PA-RISC, with the SOM
format, we could simply "cat" a bunch of object files together to make
such a package.)

Really, I can't recommend strongly enough the benefits you'd get from
taking a step back and layering your application properly. That won't
just help you with the linking step -- I think it would help you build
a more maintainable application overall.

-cary


p.s. I'd like to second what Joel said about the reply from
"anonymous". It's my hope that those kinds of comments are completely
unrepresentative of the community here. We're here to help.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]