Re: Multiple threads in one corefile on Linux--gdb support?

[Eric Paire <paire at ri dot silicomp dot fr>]

George T. Talbot wrote:
> 
> Eric Paire wrote:
> > 
> > > I've done a kernel patch for Linux which dumps multiple threads into different
> > > corefiles named core.##### where ##### is the PID.  I've been told that the
> > > ELF format'll support multiple register in a core file by having >1 PT_NOTE
> > > section.
> > >
> > > If I patch the kernel to dump the multiple register sets into a single core,
> > > will GDB with LinuxThreads support recognize the corefile without
> > > modification?
> > >
> > Two years ago, when I first developed the gdb extension for debugging
> > LinuxThreads applications, Philip Gladstone <philip@raptor.com> sent to me
> > patches for gdb-4.16 and Linux kernel in order to generate multithreaded
> > core dumps within the regular ELF format (exactly as you suggest).
> > 
> > Linus Torvalds has never included these patches into the kernel since they
> > require that the kernel kills all clones of the thread doing the core dump,
> > probably because it forces a given semantic to the cloned processes inside
> > the kernel.
> 
> Why kill all the cloned processes?  Just pause them during the core dump...
> 
AFAIK, this is something almost impossible to achieve in the current Linux
kernel, since the process must be stopped in order to get its register values,
otherwise, you will get the ones as stored the last time I entered into the
kernel. And there is no such synchronization, since unneeded for now.

> In any case, do you guys save what Linus said?  I.E. his actual objections?
> 
No, Linus has never officially sent to me any objection; you should ask
Philip Gladstone as well.

> > Last year, I have suggested another philosophy for LinuxThread core dumps
> > which does not require any kernel modification, and which I have never had
> > enough spare time to implement. Everything is done in gdb and in the
> > LinuxThread library. Here is a rough description of it:
> > 
> > 1) attach a signal handler to all signals (except SIGKILL and SIGSTOP
> >         obviously) which will be called with the sigcontext,
> > 2) on signal (generating a core dump) reception, signal all other threads,
> > 3) on each thread receiving the signal, store the context in a reserved
> >         thread-private part (usually into their own stack),
> > 4) wait for the end of all threads but one (the manager one ?)
> > 5) when only one thread remain, reset signal handler to default and resend
> >         original signal to itself, which may or may not generate a core dump,
> >         depending on the kernel implementation.
> > 
> > The kernel generates then a standard unithreaded core dump, which contains
> >         the current status of all threads.
> > 6) gdb then retreives the status of all threads by retreiving internally the
> >         status of all threads at the place it has been stored since the
> >         memory is shared between all threads, including their private stack.
> > 
> > This schema has two advantages:
> > 1) it does not require any kernel modification (which should please Linus
> >         Torvalds ;-)
> > 2) it is completely core format and processor-independent (ELF format does not
> >         cover all processors).
> > 
> > The usual comment I have received is that the status of all threads will not
> > be the one at the time the original one received the signal, since thread
> > signaling is done in user mode and other threads may have run. This is true
> > for uniprocessor machines, but it is almost impossible to have the exact
> > status of all others threads at the time the thread died on a multiprocessor,
> > since other threads may run on different processors and during the time
> > signal is handled, the other threads may have run several instructions before
> > being stopped (which is a general problem of debugging MT applications).
> > 
> > -Eric
> > P.S. nevertheless, I have kept the original patches of Philip Gladstone which
> >         I can resend to interested people.
> 
> Would it be possible to do as you suggest in user-space, and also write the
> core file with multiple register sections from user space so as to be
> compatible with debuggers ported from other platforms?
> 
This means that the LinuxThread library has to be aware of all the core
formats supported by all versions of Linux, i.e read all core sections,
insert needed information and rewrite all core sections so that the format
is coherent with the information added. I don't think that this is the
right direction for supporting multi-threaded core dumps.

The LinuxThread has now an associated library for debug interface. It is just
a matter to add new entries to support core dump management in this library.

-Eric
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ Eric PAIRE
Web  : http://www.ri.silicomp.com/~paire  | Groupe SILICOMP - Research Institute
Email: eric.paire@ri.silicomp.com         | 2, avenue de Vignate
Phone: +33 (0) 476 63 48 71               | F-38610 Gieres
Fax  : +33 (0) 476 51 05 32               | FRANCE