This is the mail archive of the cygwin mailing list for the Cygwin project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Better perl rebase solutions


For the upcoming perl-5.14.0 update I'm toying with this idea.

Problem:
Forked perl's with new XS dll's very often fail because of imagebase clashes. The current solution is perlrebase when such an error appears.
We want to automatize perlrebase whenever you install a new XS extension (via cpan).
But even when testing the new XS dll in blib/arch we might get imagebase clashes.


Idea:
Use a global $vendorarch/auto/.rebase file which stores the next available imagebase for the next to be built perl XS dll.
perlrebase also writes to this file.
I'll ship a new initial $vendorarch/auto/.rebase file.


perlrebase will add:
...
/usr/bin/rebase -v -b $baseaddr -T rebase$suff.lst | /usr/bin/tee $full
stat=$?
echo $stat
[ -n $stat ] || exit
[ -e /usr/bin/peflags.exe ] && /usr/bin/grep .dll rebase$suff.lst | /usr/bin/peflags -d0 -T - >/dev/null
/usr/bin/cat rebase$suff.lst | /usr/bin/xargs chmod g-w
/usr/bin/tail -n1 $full | \
$perl -ne'/new base = (.+), new size = (.+)/ && printf("%x\n",hex($1)+hex($2));' \
> /usr/lib/perl5/vendor_perl/$ver/$arch/auto/.rebase
/bin/cat /usr/lib/perl5/vendor_perl/$ver/$arch/auto/.rebase



ExtUtils::MakeMaker will use this new snippet:


=item dynamic_lib

Use the default to produce the *.dll's.
Add the dll size to F<$vendorarch/auto/.rebase>, which stores the
next available imagebase.

If an old dll exists and .rebase is empty, use the same rebase address
for new archdir dll's.

=cut

sub dynamic_lib {
my($self, %attribs) = @_;
my $s = ExtUtils::MM_Unix::dynamic_lib($self, %attribs);
my $ori = "$self->{INSTALLARCHLIB}/auto/$self->{FULLEXT}/$self->{BASEEXT}.$self->{DLEXT}";
my $rebase = "$self->{INSTALLVENDORARCH}/auto/.rebase";
my $imagebase;
if (-f $rebase) {
$imagebase = `/bin/cat $rebase`;
chomp $imagebase;
}
if (!$imagebase and -e $ori) {
$imagebase = `/bin/objdump -p $ori | /bin/grep ImageBase | /bin/cut -c12-`;
chomp $imagebase;
if ($imagebase gt "40000000" and $imagebase lt "80000000") {
my $LDDLFLAGS = $self->{LDDLFLAGS};
$LDDLFLAGS =~ s/-Wl,--enable-auto-image-base/-Wl,--image-base=0x$imagebase/;
$s =~ s/ \$\(LDDLFLAGS\) / $LDDLFLAGS /m;
}
} elsif ($imagebase gt "40000000" and $imagebase lt "80000000") {
my $LDDLFLAGS = $self->{LDDLFLAGS};
$LDDLFLAGS =~ s/-Wl,--enable-auto-image-base/-Wl,--image-base=0x$imagebase/ or
$LDDLFLAGS .= " -Wl,--image-base=0x$imagebase";
$s =~ s/ \$\(LDDLFLAGS\) / $LDDLFLAGS /m;
# Need a tempfile, because gmake expands $_ in the perl cmdline
open F, ">", "_rebase.pl";
print F qq(/new base = (.+), new size = (.+)/ && printf("%x\\n",hex(\$1)+hex(\$2)););
close F;
# TODO Here we create all DLL's per project with the same imagebase. We'd need
# a better tool to inc the imagebase.
$s .= "\t/bin/rebase -v -b 0x$imagebase \$@ | ";
$s .= "\$(FULLPERL) -n _rebase.pl > \$(INSTALLVENDORARCH)/auto/.rebase\n";
} else {
warn "Hint: run perlrebase to initialize $rebase\n";
}
$s;
}


The above snippet creates a GNU makefile rule how to build DLLs.

This works but has two problems:

1. There might be nameclashes with an existing _rebase.pl file in a CPAN module (very unlikely).
I started with a cmdline, like
$s .= "\t/bin/rebase -v -b 0x$imagebase \$@ | ";
$s .= "\$(FULLPERL) -ne'/new size = (.+)/ && printf(\"%x\\n\",hex(\$imagebase)+hex(\$1));)' > \$(INSTALLVENDORARCH)/auto/.rebase\n";


But I cannot solve the hex(\$1) problem. GNU make always expands $_ to /usr/bin/make and I cannot quote that make expansion. And I need to pass $_ to perl and I cannot use overlong makefile lines.
People are also using different shells. So we need to use a tempfile.
-MFile::Temp is also pretty long.


2. Multiple XS extensions per CPAN package will be created with the same imagebase. In order to check each new imagebase at run-time we'd need a better tool to increment the imagebase, which can be called from within gmake. Should I write such a beast also? But just for perl?
If so setup.exe could be extended to rebase new dlls automatically. Code-sharing.
python has the same problem, and we didn't even start with ruby.


3. The imagebase of already existing XS dll's are not reused. It is only shrinked with the next perlrebase.

perl-5.14.0 is now at 5b320000 with 74 installed extensions (typical),
with perl5.12.2 I was at 57f40000 with 94 installed extensions (medium),
with perl-5.10.1 at 5a3b0000 with 222 installed extensions (many).
perl can only use 0x57000000 - 0x6?000000.

Downwards we are limited by the active Windows system dll's, upwards by the latest rebaseall imagebase, which goes from 0x70000000 downwards.

5b320000 is a lot compared to 5a3b0000 and we will run out of space soon. I already removed all CPAN::Reporter dependencies
but the new DLL's seem to be just bigger.
--
Reini Urban
http://phpwiki.org/ http://murbreak.at/


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]