This is the mail archive of the cygwin mailing list for the Cygwin project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: Perl bug?


On Jan 22 11:26, Yitzchak Scott-Thoennes wrote:
> > Hi,
> >
> > consider the following statement:
> >
> >  $a = "a" x (100 * 1024 * 1024)
> >
> > When you create a script which does this over and over again, you'll
> > observe a strange memory problem.
> 
> Can you show your script?

#!/usr/bin/perl
$a = "a" x (100 * 1024 * 1024);
sleep 5;
$b = "b" x (100 * 1024 * 1024);
sleep 5;
$c = "c" x (100 * 1024 * 1024);
sleep 5;
$d = "d" x (100 * 1024 * 1024);
sleep 5;
$e = "e" x (100 * 1024 * 1024);
sleep 5;
$f = "f" x (100 * 1024 * 1024);
sleep 5;
$g = "g" x (100 * 1024 * 1024);
sleep 5;
$h = "h" x (100 * 1024 * 1024);
sleep 5;
$h = "h" x (100 * 1024 * 1024);
sleep 5;
$i = "i" x (100 * 1024 * 1024);
sleep 5;
$j = "j" x (100 * 1024 * 1024);
sleep 5;
$k = "k" x (100 * 1024 * 1024);
sleep 5;
$l = "l" x (100 * 1024 * 1024);
sleep 100;

> > By stracing I found that for each of these statements the following
> > happens:
> >
> >  "a"         --> malloc (2 bytes)
> >  x 100 Megs  --> realloc (100 Megs) + malloc (100 Megs)
> >
> > So the result is that each string of 100 Megs requires 200 Megs of
> > memory.  Doing this once is no problem, but doing it over and over
> > again will hit the maximum memory available twice as early.
> 
> This is as I would expect.  Most operators have a target, a temporary
> lexical, allocated to store their results.  Like all lexicals, these
> hold on to any memory they have allocated in the hope of saving having
> to allocate it a second time.

The problem is that it's *not* reused.  If you use strace when running
this script you see the allocations as I described them.  When the
2 Gigs virtual memory size for the process are used up, mmap (which
is called by malloc for big memory chunks) is called and returns -1,
MAP_FAILED.  Then malloc tries to get the memory from the heap, when
that fails, it just prints "Out of memory!", munmaps half of the above
allocations and then exits.

Just call strace and you'll see.

> > I can only assume that either the garbage collector doesn't kick in when
> > it should, or the garbage collector doesn't even know about this wasted
> > memory, which would be a generic memory leak either way.  However, it's
> > not clear if this depends on the Perl version, or if it depends on the
> > 64 bit int setting when building Perl.
> 
> I'd more likely suspect differences in the usemymalloc setting (perl
> -V:usemymalloc).  I'm not clear on exactly what you are seeing; again,

All three perl's on all three systems use the system malloc, not the
builtin malloc implementation (usemymalloc='n').


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]