This is the mail archive of the
gdb@sourceware.org
mailing list for the GDB project.
Re: Debugging issue with gdbserver and a daemon on the target
- From: Laszlo Papp <lpapp at kde dot org>
- To: Pedro Alves <palves at redhat dot com>
- Cc: gdb at sourceware dot org
- Date: Tue, 19 Aug 2014 19:01:29 +0100
- Subject: Re: Debugging issue with gdbserver and a daemon on the target
- Authentication-results: sourceware.org; auth=none
- References: <CAOMwXhPxtfv7f=akjpuTh+HNCGDeEm0vxen1Kp3pPco++gvuLg at mail dot gmail dot com> <53F3743F dot 3000106 at redhat dot com> <CAOMwXhMmyXmHi9gZADtkTYb9mbF2hZRv2vHPC5AEcNWkE9duBg at mail dot gmail dot com> <53F3776D dot 2010705 at redhat dot com>
On Tue, Aug 19, 2014 at 5:12 PM, Pedro Alves <palves@redhat.com> wrote:
> On 08/19/2014 05:02 PM, Laszlo Papp wrote:
>> On Tue, Aug 19, 2014 at 4:58 PM, Pedro Alves <palves@redhat.com> wrote:
>>> On 08/19/2014 04:44 PM, Laszlo Papp wrote:
>>>
>>>>
>>>> gdbserver --attach 192.168.0.32:2345 pid-of-my-daemon
>>>>
>>>
>>>> (gdb) bt
>>>> #0 0x44ad26ec in select () at ../sysdeps/unix/syscall-template.S:81
>>>> #1 0x0002ac08 in bar (timeout=10, name=0x42f30 <yy_ec+56> "foo") at
>>>> src/socket.c:906
>>>> #2 0x0003284c in main (argc=0, argv=0x0) at src/bar.c:679
>>>> (gdb)
>>>>
>>>>
>>>> ... And then I do some communication with the daemon where the foo
>>>> function is executed based on the logs, but the breakpoint is not hit.
>>>> I wished to try hardware breakpoints, but they are not presented on my
>>>> hardware.
>>>>
>>>> Furthermore, if I use the same workflow on a binary that is
>>>> "one-shot", i.e. not running continuously as a daemon, the debugging
>>>> workflow for stopping at main works with exactly the aforementioned
>>>> software breakpoint issue.
>>>>
>>>> I am completely clueless at this point. Do you know how I can debug a
>>>> daemon with gdbserver?
>>>
>>> "daemon" and "select" makes me think "fork". If the daemon is handling
>>> requests by forking a child, and then it's the child that calls 'foo',
>>> then this is expected, as GDBserver doesn't know how to follow forks
>>> currently. It's WIP, patches have been posted. Meanwhile, the usual
>>> thing to do it to attach to the child process the daemon spawns instead
>>> of the main daemon pid. You'll usually do that by adding a busyloop in
>>> the child somewhere, like:
>>>
>>> volatile int gdb_here;
>>>
>>> while (!gdb_here)
>>> sleep (1);
>>>
>>> and after attaching to the child, do "print gdb_here = 1; continue".
>>>
>>> Thanks,
>>> Pedro Alves
>>
>> Thanks Pedro for the prompt reply. Unfortunately, I am already
>> attaching to the child right after the fork. I wonder if this can
>> happen if some source file was missing?
>
> It shouldn't. Source files are only used for display. Where to
> place a breakpoint is derived from the debug info in the binary.
>
> I'd suggest just trying to step through the code instead of
> putting a break at "foo", and see if that much works. On an
> arm system, stepping is actually implemented with magic
> breakpoints behind the scenes.
>
>> Btw:
>>
>> gdbserver --version
>> GNU gdbserver (GDB) 7.5.1
>>
>> arm-polatis-linux-gnueabi-gdb --version
>> GNU gdb (GDB) 7.5.1
>>
>
> Knee-jerk reaction is to suggest a more recent GDB/GDBserver. Note
> building these isn't very hard. There aren't that many
> dependencies.
>
> Thanks,
> Pedro Alves
Hmm, it seems that the stripped binary on the target and the one on
the host were out-of-sync. This is really strange since I have not
changed the source code. Seems different compilations still can get
out-of-sync for the same code so that when I rebuild the same source
code, I always need to update the binary on the target, too?
Anyway, now I only have problems with finding the sources file to view
them in cgdb. I do not know why it is wrong, but it seems to be. As
you can see the paths are set up for dwarf correctly:
readelf --debug-dump
./tmp/work/foo-foo-linux-gnueabi/foo-core-image-dbg/1.0-r0/rootfs/usr/bin/.debug/foo
| grep DW_AT_comp_dir
<35> DW_AT_comp_dir :
/usr/src/debug///////////////////////////////////////////////////////////////////////////////////eglibc/2.17-r3/eglibc-2.17/libc/csu
<df> DW_AT_comp_dir : (indirect string, offset: 0x31):
/usr/src/debug/eglibc/2.17-r3/eglibc-2.17/libc/csu
<183> DW_AT_comp_dir :
/usr/src/debug///////////////////////////////////////////////////////////////////////////////////eglibc/2.17-r3/eglibc-2.17/libc/csu
<22d> DW_AT_comp_dir : (indirect string, offset: 0x749):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/bar
<102f> DW_AT_comp_dir : (indirect string, offset: 0x749):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/bar
<2e98> DW_AT_comp_dir : (indirect string, offset: 0x749):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/bar
<3e34> DW_AT_comp_dir : (indirect string, offset: 0x749):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/bar
<471a> DW_AT_comp_dir : (indirect string, offset: 0x749):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/bar
<4c5f> DW_AT_comp_dir : (indirect string, offset: 0x749):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/bar
<55b9> DW_AT_comp_dir : (indirect string, offset: 0x749):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/bar
<84b8> DW_AT_comp_dir : (indirect string, offset: 0x749):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/bar
<94d1> DW_AT_comp_dir : (indirect string, offset: 0x749):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/bar
<a1e9> DW_AT_comp_dir : (indirect string, offset: 0x749):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/bar
<af53> DW_AT_comp_dir : (indirect string, offset: 0x3874):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/baz
<c382> DW_AT_comp_dir : (indirect string, offset: 0x3874):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/baz
<c952> DW_AT_comp_dir : (indirect string, offset: 0x3874):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/baz
<d281> DW_AT_comp_dir : (indirect string, offset: 0x3874):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/baz
<dcc0> DW_AT_comp_dir : (indirect string, offset: 0x3874):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/baz
<e093> DW_AT_comp_dir : (indirect string, offset: 0x3874):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/baz
<e953> DW_AT_comp_dir : (indirect string, offset: 0x3874):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/baz
<efd6> DW_AT_comp_dir : (indirect string, offset: 0x3874):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/baz
<fb07> DW_AT_comp_dir : (indirect string, offset: 0x3874):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/baz
<10cd9> DW_AT_comp_dir : (indirect string, offset: 0x3874):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/baz
<127e6> DW_AT_comp_dir : (indirect string, offset: 0x3874):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/baz
<132ab> DW_AT_comp_dir : (indirect string, offset: 0x3874):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/baz
<13bf3> DW_AT_comp_dir : (indirect string, offset: 0x585a):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/meh
<1556d> DW_AT_comp_dir : (indirect string, offset: 0x585a):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/meh
<162d4> DW_AT_comp_dir : (indirect string, offset: 0x585a):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/meh
<17481> DW_AT_comp_dir : (indirect string, offset: 0x585a):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/meh
<17900> DW_AT_comp_dir : (indirect string, offset: 0x585a):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/meh
<18481> DW_AT_comp_dir : (indirect string, offset: 0x585a):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/meh
<1903f> DW_AT_comp_dir : (indirect string, offset: 0x585a):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/meh
<19432> DW_AT_comp_dir : (indirect string, offset: 0x585a):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/meh
<19524> DW_AT_comp_dir : (indirect string, offset: 0x585a):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/meh
<19fe1> DW_AT_comp_dir : (indirect string, offset: 0x585a):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/meh
<1d2b4> DW_AT_comp_dir : (indirect string, offset: 0x585a):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/meh
<1dc34> DW_AT_comp_dir : (indirect string, offset: 0x585a):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/meh
<1f2c1> DW_AT_comp_dir : (indirect string, offset: 0x585a):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/meh
<1fa4f> DW_AT_comp_dir : (indirect string, offset: 0x585a):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/meh
<1fe13> DW_AT_comp_dir : (indirect string, offset: 0x585a):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/meh
<203e4> DW_AT_comp_dir : (indirect string, offset: 0x7cfa):
/usr/src/debug/flex/2.5.35-r3/flex-2.5.35
<2042a> DW_AT_comp_dir : (indirect string, offset: 0x31):
/usr/src/debug/eglibc/2.17-r3/eglibc-2.17/libc/csu
<2056c> DW_AT_comp_dir : (indirect string, offset: 0x7de0):
/usr/src/debug/eglibc/2.17-r3/eglibc-2.17/libc/io
<20833> DW_AT_comp_dir :
/usr/src/debug///////////////////////////////////////////////////////////////////////////////////eglibc/2.17-r3/eglibc-2.17/libc/csu
DW_AT_comp_dir DW_FORM_string
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_string
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_strp
DW_AT_comp_dir DW_FORM_string
... yet, gdb says src/bar.c cannot be found even though it should be
in the aforementioned
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/meh
path, provided my sysroot setting is good above, but if that was not
good, it would not load the binaries anyway, right? So, if I set that
path one line above with the "-d" option to gdb, then the source file
can be viewed. What may be going on here?
Thanks in advance. I am so desperately lost. :(