Re: PATCH: fail to improve psymtab memory consumption

[Daniel Berlin <dan at cgsoftware dot com>]

Jim Blandy <jimb@zwingli.cygnus.com> writes:

> Daniel Berlin <dan@cgsoftware.com> writes:
>> Hence the reason I only read the part of the various sections for a
>> given CU (rather than reading the entire section).
> 
> Yeah, I think that's the way to go.  The 12Mb of .debug_info,
> .debug_abbrev, and .debug_line data lying there is the biggest issue.
> There's no reason we couldn't read it in for partial symbol table
> construction, throw it away when we're done, and then bring in data
> for only those CU's we need, only when we need it.
> 
> GNU malloc uses mmap for big chunks of data, so when you free a big
> block of memory (like debug_info_buffer) after doing the partial
> symbol scan, the memory really does go back to the system; it doesn't
> just get added to malloc's free list.
> 
>> I actually use mmap when possible, but that's for speed, rather than
>> memory savings.
> 
> Is it really faster?
Yes, because it saves buffer copying (I think, it's 2am, which is
about the time i usually say something stupid, so feel free to smack
me around if i'm wrong)
(You don't have to dandy about copying buffers into the right
place. With an mmap'd area, it's already *in* the right place.)


Nathan myers confirms my suspicion, as well.

http://pgsql.profnet.pl/mhonarc/pgsql-hackers/2001-05/msg01233.html
"
Using mmap() in place of disk read() almost always results in enough
performance improvement to make doing so worth a lot of disruption."
"

Everything i can pull up on google says mmap is so much faster than
malloc+fread that it's not even funny, which means it's not just that
it feels faster.  On a 100 meg debug info file, you can easily notice
the difference.

I hacked up bfd to use mmap when possible, and linking got a *whole
lot* faster on large files.  :).



> 
>> It doesn't buy us anything if we still mmap the entire section, and
>> then touch every part. :)
> 
> Which we do when in building the partial symbol table.
Right.

However, since we lazily read in full symbols, we don't have this
behavior unless you force readin of all the symbols. You can do this
with maint check symtabs, or using the pathological unchecked for case
of find_pc_sect_symtab with a pc of 0. We should immediately return
NULL in that case, since nothing contains the symbol table for pc 0,
but instead, we convert every psymtab to symtab looking for the symtab
for a plainly invalid memory address. We of course, never find it, and
return NULL anyway, but not before wasting a ton of time and memory.
Whoops.
This patch will fix that (It's too late to make up a simple changelog
entry for it, i'm about to fall down on the keyboard).

*************** find_pc_sect_symtab (CORE_ADDR pc, asect
*** 1374,1379 ****
--- 1355,1362 ----
    register struct objfile *objfile;
    CORE_ADDR distance = 0;
 
+   if (pc == 0)
+     return NULL;
    /* Search all symtabs for the one whose file contains our address, and which
       is the smallest of all the ones containing the address.  This is designed
       to deal with a case like symtab a is at 0x1000-0x2000 and 0x3000-0x4000



-- 
"They say we're 98% water.  We're that close to drowning...
(Picks up his glass of water from the stool...)  I like to live
on the edge...
"-Steven Wright