Re: Inter-CU DWARF size optimizations and gcc -flto

[Daniel Jacobowitz <drow at false dot org>]

On Fri, Mar 2, 2012 at 9:54 PM, Tom Tromey <tromey@redhat.com> wrote:
>>>>>> "Daniel" == Daniel Jacobowitz <drow@false.org> writes:
>
> Daniel> You are correct, it does crush GDB :-) ?I routinely try - emphasis on
> Daniel> try - to use GDB on programs with between 2500 and 5500 shared
> Daniel> libraries. ?It's agonizing. ?I have another project I want to work on
> Daniel> first, and not much time for GDB lately, but this is absolutely on my
> Daniel> list to improve.
>
> I am curious how you plan to improve it.

I have no idea.  One thing I'd like to revisit is your work on
threaded symbol load; I have plenty of cores available, and the
machine is pretty much useless to me until my test starts.  There's
also a lot of room for profiling to identify bad algorithms; I think
we spend a lot of time reading the solib list from the inferior
(something I thought I and others had fixed thoroughly already...) and
I routinely hit inefficient algorithms e.g. during "next".

>
>
> The plan I mentioned upthread is probably pretty good for scaling to
> distro-sized programs, say 200 shared libraries or less (this is
> LibreOffice or Mozilla). ?Maybe we could get a bit more by putting
> minsyms into the index.
>
> I am not so confident it would let gdb scale to 5000 shared libraries
> though.
>
> For that size I've had two ideas.
>
> First, and simplest, punt. ?Make the user disable automatic reading of
> shared library debuginfo (or even minsyms) and make the user explicitly
> mention which ones should be used -- either by 'sharedlibrary' or by a
> linespec extension.
>
> I guess this one would sort of work today. ?(I haven't tried.)

I am hugely unexcited by this.  Even if did basic usability work on
top of that - e.g. automatically load all solibs that appear in the
backtrace - the inability to find sources by file:line is a huge
problem for me.

>
>
> Second, and harder, is the "big data" approach. ?This would be something
> like -- load all the debuginfo into a server, tagged by build-id,
> ideally with global type- and symbol-interning; then change gdb to send
> queries to the server and get back the minimal DWARF (or DWARF-esque
> bits) needed; crucially, this would be a global operation instead of
> per-objfile, so that gdb could exploit parallelism on the server side.
>
> Parallelism seems key to me. ?Parallelism on the machine running gdb
> probably wouldn't work out, though, on the theory that there'd be too
> much disk contention. ?Dunno, maybe worth trying.

This is an idea I'm excited by.  It works well along with Cary's
http://gcc.gnu.org/wiki/DebugFission, too; a separate process could
handle the changes as individual shared libraries are rebuilt.

Something I've been thinking about is that incrementalism is hard in
GDB because the symbol tables are so entwined... adding any sort of
client/server interface would force us to detangle them, and then
individual objects could have a longer life.

-- 
Thanks,
Daniel