This is the mail archive of the systemtap@sourceware.org mailing list for the systemtap project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Missing unwind data for module


Hi,

I am attempting to probe the kernel function neigh_create(). When I staprun run the kernel module produced by stap on the target system, I get this WARNING and partial backtrace:

[root@s01b08 jharan]# staprun neigh_create_enter.ko
neigh_create_call began, arp_tbl 0xffffffff81b16540
WARNING: Missing unwind data for module, rerun with 'stap -d insane'
neigh_create() tbl 0xffffffff81b16540, dev 0xffff8806303c5020, name bond1.3091
neigh_create() hit 0, dev 0xffff8806303c5020, name bond1.3091, addr 0xa9fe4014, addr_string 169.254.64.20
neigh_create() IP address matches
 0xffffffff814461c0 : neigh_create+0x0/0x520 [kernel]
 0xffffffff8149cdf2 : arp_find+0x1a2/0x230 [kernel]
 0xffffffff81457db3 : eth_rebuild_header+0x73/0x80 [kernel]
 0xffffffffa013a3bf [insane]
 0x0 (inexact)

insane is a kernel module that I got off of a website, so it's not part of the RHEL63 distro that the rest of the kernel came from. (Note: it's a virtual device driver that is used to generating packet drops). I assume that the above is related to some sort of symbol information in insane.ko being missing either when I build the module or when I run stap to build neigh_create_enter.ko.

If I follow the suggestion in the WARNING message when I build neigh_create_enter.ko on the build machine, I get this:

[root@s01b06 jharan]# stap -g -v -r 2.6.32-279.5.1.el6.x86_64 neigh_create_enter.stp -m neigh_create_enter -p 4 -d insane.ko
Pass 1: parsed user script and 82 library script(s) using 97408virt/23032res/2960shr kb, in 120usr/0sys/131real ms.
Pass 2: analyzed script: 2 probe(s), 10 function(s), 2 embed(s), 2 global(s) using 319216virt/112628res/4248shr kb, in 1150usr/160sys/1311real ms.
Pass 3: translated to C into "/tmp/stap31Gjb1/neigh_create_enter_src.c" using 318840virt/117260res/8972shr kb, in 460usr/70sys/528real ms.
neigh_create_enter.ko
Pass 4: compiled C into "neigh_create_enter.ko" in 8430usr/490sys/8909real ms.
[root@s01b06 jharan]#

No warning generated, but when I run the result on the target machine, I still get the warning and partial backtrace:

[root@s01b08 jharan]# staprun neigh_create_enter.ko
neigh_create_call began, arp_tbl 0xffffffff81b16540
WARNING: Missing unwind data for module, rerun with 'stap -d insane'
neigh_create() tbl 0xffffffff81b16540, dev 0xffff8806303c5020, name bond1.3091
neigh_create() hit 0, dev 0xffff8806303c5020, name bond1.3091, addr 0xa9fe4014, addr_string 169.254.64.20
neigh_create() IP address matches
 0xffffffff814461c0 : neigh_create+0x0/0x520 [kernel]
 0xffffffff8149cdf2 : arp_find+0x1a2/0x230 [kernel]
 0xffffffff81457db3 : eth_rebuild_header+0x73/0x80 [kernel]
 0xffffffffa013a3bf [insane]
 0x0 (inexact)
^C[root@s01b08 jharan]#

Note that in other call trees to neigh_create() I get nice clean back traces from the system call on down:

neigh_create() hit 261, dev 0xffff88066f7dd020, name bond1.3092, addr 0xa9fe4014, addr_string 169.254.64.20
neigh_create() IP address matches
 0xffffffff814461c0 : neigh_create+0x0/0x520 [kernel]
 0xffffffff8149b8ee : arp_bind_neighbour+0xbe/0xc0 [kernel]
 0xffffffff8146d2e7 : rt_intern_hash+0x147/0x590 [kernel]
 0xffffffff8146db6d : ip_route_output_slow+0x43d/0x9a0 [kernel]
 0xffffffff8146e26b : __ip_route_output_key+0x19b/0x1b0 [kernel]
 0xffffffff81496244 : ip4_datagram_connect+0x184/0x320 [kernel]
 0xffffffff814a1c7c : inet_dgram_connect+0x2c/0x80 [kernel]
 0xffffffff81428de7 : sys_connect+0xd7/0xf0 [kernel]
 0xffffffff8100b0f2 : system_call_fastpath+0x16/0x1b [kernel]
 0x0 (inexact)

The problem seems to occur only when the call tree goes thru the insane.ko module.

Suggestions as to what I should do to avoid the warning and get a full stack trace most appreciated. My stap script follows:

[root@s01b06 jharan]# cat neigh_create_enter.stp
global arp_tbl

%{
#include <net/arp.h>
%}
function the_table_address:long () %{ /* unmangled */
        /* rely on EXPORT_SYMBOL ... to let this resolve */
        THIS->__retvalue = (int64_t) & arp_tbl;
%}
probe begin {
        arp_tbl = the_table_address()
        printf("neigh_create_call began, arp_tbl %p\n", arp_tbl)
        /* printf("%d\n", @cast(the_table_address(),"neigh_table","kernel")->family) */
}

global hit
probe kernel.function("neigh_create").call
{
        tbl = $tbl
        dev_name = kernel_string($dev->name)
        printf("neigh_create() tbl %p, dev %p, name %s\n", tbl, $dev, dev_name)
        if (tbl == arp_tbl) {
                ip_addr_network = kernel_int($pkey)
                ip_addr_string = ip_ntop(ip_addr_network)
                printf("neigh_create() hit %d, dev %p, name %s, addr 0x%x, addr_string %s\n",
                        hit++, $dev, dev_name, ntohl(ip_addr_network), ip_addr_string)
                if (ntohl(ip_addr_network) == 0xa9fe4014) {
                        printf("neigh_create() IP address matches\n")
                        print_backtrace()
                }
        }
}
[root@s01b06 jharan]#

I'm using systemtap version 1.7-5 from the RHEL63 distro.

Thanks,

Jeff Haran


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]