The Black Art of DLL Creation (revisited)

[Soren A <soren_andersen at fastmail dot fm>]

Is it a science, or an art?

[The following description pertains to the *July 2002* version of dlltool].

I am wanting to know about something relating to building of DLLs on
Cygwin using the GNU Binutils tools. Up until this point I've been happy
enough just letting recent GCC (ld) versions work their magick with the
-shared switch, but my recent experimentation (described below) led me
to re-visit the venerable (and deprecated, and despised?) "dlltool". 

This message is really a long-delayed re-taking-up of questions I had 
responded to by (mainly) Charles Wilson in the past two or three years.

The scenario is that I am trying to re-create the process of building 
bleadperl (experimental branch Perl [5.9.0] from the ActiveState-hosted 
[canonical] host location). But it doesn't really matter that it is perl -- 
it could be anything. That it is Perl is not of the essence. But anyway, I 
want the building of Perl to be conducted the way *I* think is rational. 
Taking apart the current build setup, we see that dlltool (through dllwrap) 
is invoked for building the shared Perl library (the bulk of the guts of 
perl on Cygwin and Win32 generally, is placed into a DLL; the executable is 
only ca. 50KB in size and acts as a mere "invoker"). It is well known that 
the functionality to directly create a DLL from object modules has been 
part of GCC(ld) for quite a long time now, but one thing that you can get 
from using dlltool, that you don't (I think?) get from using 'gcc -shared' 
is a .DEF file that is basically correct. If you ask 'gcc -shared' for a 
.DEF output using the appropriate switches, you get included stuff like a 
.bss[data?] that isn't supposed to be exported.

Anyway, some opinions would hold that the current way Perl is built on 
Cygwin is an anachronism and is begging to be updated. Dllwrap takes much 
longer on my system to create the DLL than invoking 'gcc -shared' does, for 
one thing. BUT, I would like (for reasons that may not seem important to 
others) to have a valid (by which I mean 'containing all symbols [functions 
and global data] that need to be exported [visible to external exes] by the 
DLL, and none that do not') .DEF file to work with. (OK, one reason is that 
I am interested in the internals of Perl, and examining the contents of the 
.DEF file gives a glimpse into those internals).

So I ran some experiments. I used a command (in the Makefile) like this, to 
generate a .DEF file:

  dlltool --kill-at -z [mylibbasename].def [objfiles1] [objfiles2] [...]

And I then invoked (in the next step) GCC like so:

  gcc -shared [objfiles1] [objfiles2] [...] [mylibbasename].def

Observations on what happened:

  1. The .DEF output by dlltool isn't acceptable to ld(1). It dumps
     core. The problem is the first line of the .DEF file generated by
     dlltool. This first line is commented and is a record of the
     invocation used to create the .DEF.

  2. Only a handful of symbols were listed under EXPORTS. These symbols
     were all prefixed "XS_[foo]" and it is possible, based on my
     examination of the source, that these functions were declared in
     some way that doesn't involve "__declspec(dllexport)". The
     dozens of other symbols that one expects and needs to see weren't
     exported. 

  3. The -k (--kill-at) switch did nothing. Ordinals were still being
     postpended to each symbol's entry.

Fixing (1) and (3) was accomplished by post-processing the .DEF file using 
sed(1). The resulting .DEF file was acceptable to ld(1) ['gcc -shared'] but 
there was still the problem of (2) unresolved.

I made sure that the Perl macros "EXT" and "cEXT" (constant) etc. were
being properly substituted with "__declspec(dllexport)". The header in
the cygwin/ subdir of the Perl src tree causes it NOT to be set, --
Gerrit and others who are knowledgeable about CygwinPerl may note this
-- but I found a way to override that. 

Backing up: in theory TTBOMU, if I've thus marked a symbol for export
this way, the Cygwin port of GNU ld(1) *should* export it
unconditionally. Furthermore, TTBOMU, this SHOULD only be necessary for
global variables (data) in the first place -- all *functions* should be
exported anyway(?). In cases where the src package has no provisions for
this, we've used the "--export-all" switch (for either dlltool OR 'gcc
-shared') to cause eveything but the special Cygwin excluded symbols
relating to entry points, etc. to be marked for export in the final DLL.
This is what the CygwinPerl build setup does at present, and I don't
understand why. The mechanism for marking symbols with
"__declspec(dll[ex|in]port" is already part of Perl src. Why can't we
use it? Clearly *somebody* does or did -- probably MSVC does. 

So my primary question is now, is it documented behavior for dlltool(1)
to ignore "__declspec(dllexport)" when creating a list of symbols for
export (which it writes out to a .DEF file)? If not, is it broken now?
If so, WHY? BTW, I ran these tests using the *July 2002* version of
dlltool. Just in the last few minutes I've downloaded the latest
Binutils but it will be as much as several hours before I've reset my
Perl build setup to try another run. 

When I reran the dlltool as shown above but with the "--export-all" flag
added, I got a .DEF that included everything. If I feed that .DEF (after
repairing it by post-processing with sed) INTO 'gcc -shared [...]' I get
a good perl dll and import / interface lib (FWIW I greatly prefer the
latter terminology). Joy-joy.

Sorry if this is a known dlltool bug or something. Even if so, hopefully
this little journal of my adventure might benefit some future
mail-archive seeker. 

   Best,
    Soren


-- 
Yes, it's really Sören, not Soren.




--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/