This is the mail archive of the systemtap@sourceware.org mailing list for the systemtap project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

what a probe-induced kernel fault looks like


Hi -

Just as a matter of curiosity, here's what a kernel fault looks like
when a division-by-zero is triggered (by virtue of incomplete checks).
What's interesting about it is that the erroneous division was inside
a "begin" probe.  Due to a questionable aspect of runtime design, this
is run long after actual module load/init time: it happens after an
explicit handshake with stpd.

This sort of handshaking-based protocol would be terribly
inappropriate in the case of module onloading.  If stpd dies, there
would be no way to safely remove the module, e.g. to trigger orderly
kprobes removals.  Martin, how does shutdown happen in your model?

For what it's worth, in my mental model of probe shutdown/startup, the
translator-emitted code would own the module init/exit hooks; perform
begin/end probes and kprobes registrations therein, and call into the
runtime to begin whatever stpd chitchat it wishes to engage in.  The
probe would in no way rely in the existence or activity of stpd for
its crucial life cycle management functions.


Aug 11 21:58:34 elastic kernel: divide error: 0000 [#1]
Aug 11 21:58:34 elastic kernel: SMP
Aug 11 21:58:34 elastic kernel: Modules linked in: stap_500_1123811911(U) stp_control(U) relayfs(U)
vfat fat sch_ingress cls_u32 sch_sfq sch_htb ip_nat_ftp ip_conntrack_ftp cpufreq_stats snd_seq_midi
snd_seq_midi_event snd_seq snd_mpu401 snd_mpu401_uart snd_rawmidi snd_seq_device i8xx_tco radeon drm ipt_state ipt_REJECT ipt_LOG ipt_limit iptable_mangle iptable_nat ip_conntrack iptable_filter ip_tables parport_pc lp parport p4_clockmod microcode dm_mod video button battery ac usb_storage uhci_hcd ehci_hcd hw_random snd_intel8x0 snd_ac97_codec snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd soundcore snd_page_alloc sk98lin epic100 mii floppy ext3 jbd raid1 ata_piix libata sd_mod scsi_mod
Aug 11 21:58:34 elastic kernel: CPU:    1
Aug 11 21:58:34 elastic kernel: EIP:    0060:[<f9101ab2>]    Tainted: GF     VLI
Aug 11 21:58:34 elastic kernel: EFLAGS: 00210286   (2.6.12-1.1372_FC3.rootsmp)
Aug 11 21:58:34 elastic kernel: EIP is at probe_0+0x16c/0x377 [stap_500_1123811911]
Aug 11 21:58:34 elastic kernel: eax: 80000000   ebx: 00000000   ecx: ffffffff   edx: ffffffff
Aug 11 21:58:34 elastic kernel: esi: f9105ee8   edi: 00000000   ebp: 00000000   esp: ea741ccc
Aug 11 21:58:34 elastic kernel: ds: 007b   es: 007b   ss: 0068
Aug 11 21:58:34 elastic kernel: Process stpd (pid: 6006, threadinfo=ea741000 task=f425b020)
Aug 11 21:58:34 elastic kernel: Stack: badc0ded f9105ed4 f9105ed4 00001776 d12ca210 00000005 f9101d38 00000158
Aug 11 21:58:34 elastic kernel:        00000158 f4d73b00 00000000 d12ca210 00001776 f9101d9d f9101151 00000000
Aug 11 21:58:34 elastic kernel:        f9101306 f91012bb f8c2d157 d12ca200 f6884880 00000001 f8c2d135 f8c2d1d9
Aug 11 21:58:34 elastic kernel: Call Trace:
Aug 11 21:58:34 elastic kernel:  [<f9101d38>] enter_0_0+0x7b/0xd1 [stap_500_1123811911]
Aug 11 21:58:34 elastic kernel:  [<f9101d9d>] systemtap_module_init+0xf/0x25 [stap_500_1123811911]
Aug 11 21:58:34 elastic kernel:  [<f9101151>] _stp_handle_start+0x8/0x2c [stap_500_1123811911]
Aug 11 21:58:34 elastic kernel:  [<f9101306>] _stp_cmd_handler+0x4b/0x4f [stap_500_1123811911]
Aug 11 21:58:34 elastic kernel:  [<f91012bb>] _stp_cmd_handler+0x0/0x4f [stap_500_1123811911]
Aug 11 21:58:34 elastic kernel:  [<f8c2d157>] _stp_ctrl_handler+0x22/0x27 [stp_control]
Aug 11 21:58:34 elastic kernel:  [<f8c2d135>] _stp_ctrl_handler+0x0/0x27 [stp_control]
Aug 11 21:58:34 elastic kernel:  [<f8c2d1d9>] _stp_msg_rcv_skb+0x69/0x86 [stp_control]
Aug 11 21:58:34 elastic kernel:  [<f8c2d20d>] _stp_msg_rcv+0x17/0x50 [stp_control]
Aug 11 21:58:34 elastic kernel:  [<c02b7c16>] netlink_data_ready+0x51/0x53
Aug 11 21:58:34 elastic kernel:  [<c02b711b>] netlink_sendskb+0x19/0x35
Aug 11 21:58:34 elastic kernel:  [<c02b7967>] netlink_sendmsg+0x265/0x2d3
Aug 11 21:58:34 elastic kernel:  [<c011d0d9>] __wake_up_common+0x35/0x55
Aug 11 21:58:34 elastic kernel:  [<c029aa99>] sock_sendmsg+0xf3/0x10e
Aug 11 21:58:34 elastic kernel:  [<c011d12b>] __wake_up+0x32/0x43
Aug 11 21:58:34 elastic kernel:  [<c021686d>] n_tty_receive_buf+0xea/0xe4c
Aug 11 21:58:34 elastic kernel:  [<c021686d>] n_tty_receive_buf+0xea/0xe4c
Aug 11 21:58:34 elastic kernel:  [<c0152ba8>] do_no_page+0x77/0x31a
Aug 11 21:58:34 elastic kernel:  [<c0134133>] autoremove_wake_function+0x0/0x37
Aug 11 21:58:34 elastic kernel:  [<c0161835>] fget+0x3b/0x42
Aug 11 21:58:34 elastic kernel:  [<c029bd88>] sys_sendto+0xc7/0xe2
Aug 11 21:58:34 elastic kernel:  [<c013402a>] remove_wait_queue+0xf/0x34
Aug 11 21:58:34 elastic kernel:  [<c0218188>] write_chan+0x1ac/0x1fa
Aug 11 21:58:34 elastic kernel:  [<c011d12b>] __wake_up+0x32/0x43
Aug 11 21:58:35 elastic kernel:  [<c029bdda>] sys_send+0x37/0x3b
Aug 11 21:58:35 elastic kernel:  [<c029c5a4>] sys_socketcall+0x172/0x286
Aug 11 21:58:35 elastic kernel:  [<c0103fd9>] syscall_call+0x7/0xb
Aug 11 21:58:35 elastic kernel: Code: 00 00 00 00 00 80 c7 86 94 00 00 00 ff ff ff ff 09 c1 89 86 98 00 00 00 89 96 9c 00 00 00 0f 84 75 01 00 00 8b 86 90 00 00 00 99 <f7> be 98 00 00 00 89 d0 99 89 86 80 00 00 00 89 46 18 89 96 84


- FChE


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]