Test suite results for ARM with uprobes

[Wade Farnsworth <wade_farnsworth at mentor dot com>]

Hi all,

I ran the systemtap test suite on a beagleboard with uprobes support 
enabled.  However, I'm running into several errors.

My configuration is as follows:
* systemtap git rev f52d32a9f57d228627ee08e39f0bbcf3f3faae20.
* kernel 2.6.37.6
* gcc is 4.5.1

The full logs may be found at:

http://dl.dropbox.com/u/40714612/stap-logs-beagleboard-20111206.tar.gz

I realize this is quite a large list to work through, but I would be 
very appreciative if anyone could shed any light on them.  If there's 
anything obvious that I'm missing I'd be glad to hear it.  Otherwise, 
I'll continue digging at them.

First, four tests are causing hangs, panics or other kernel errors.  I 
had to disable these tests in order to proceed further:

* systemtap.base/pr10854.exp results in the following:
INIT: Id "S" respawning too fast: disabled for 5 minutes
Kernel panic - not syncing: Attempted to kill init!

* systemtap.base/itrace.exp will hang in run_ls_1_sec at the line "catch 
{wait -i $exe_id}".  It looks like the ls process never exits, so we 
just spin on this line.  If I comment out this line, I hit the kernel 
BUG in linux/kernel/exit.c:forget_original_parent() followed by a NULL 
pointer dereference in do_exit().

* systemtap.examples/check.exp gets an "Unable to handle kernel paging 
request at virtual address" error

* systemtap.unprivileged/unprivileged_myproc.exp hits the BUG at 
runtime/uprobes2/uprobes.c:uprobe_free_task()

I'm guessing that there may be some bugs in the ARM uprobes 
implementation, but I haven't yet tracked these down any further.

Once these tests have been disabled, the rest of the test suite will 
complete.  Here are some of the failures that I'm not sure about and 
some relevant notes:

FAIL: cast-scope-m32-O
FAIL: cast-scope-m32-O2
[WARNING: Can't parse SDT_V3 operand '[fp,': identifier '$arg1' at 
/home/root/stuff/systemtap/testsuite/systemtap.base/cast-scope.stp:15:32]

FAIL: cxxclass-m32
FAIL: cxxclass-m32-O
FAIL: cxxclass-m32-O2
[Got "semantic error: unable to find local 'arg1' near pc 0x842c  in 
<unknown> 
/home/root/stuff/systemtap/testsuite/systemtap.base/cxxclass.cxx ( 
(alternatives: $i $inst): identifier '$arg1' at 
/home/root/stuff/systemtap/testsuite/systemtap.base/cxxclass.stp:13:24"]

FAIL: debugpath-good (eof) [This one puzzles me.  I've built with 
CONFIG_DEBUG_INFO enabled and the build directory exists in 
/lib/modules/`uname -r`/build.  Is there something else that I need to 
do to get systemtap access to the debug info?]

FAIL: flightrec2 (log file size (5 != 3 + 3)) [stat: cannot stat 
`flightlog.out.1': No such file or directory]

FAIL: global_end (11) [Looks like the queue stats being taken in 
global_end2.stp is not occurring]

FAIL: gtod (0) [events marked "kern" are occurring before events marked 
"appl"]

FAIL: inlinedvars-m32-O
FAIL: inlinedvars-m32-O2
[line 1: expected "call (22,84)"
Got "  (84,22)"]

FAIL: library sdt_misc * (0 != 15)
FAIL: library sdt_misc *libsdt* (0 != 15)
FAIL: library sdt_misc ./libsdt.so (0 != 15)
FAIL: library printf --ldd (0) (0 != 4)
[Looks like systemtap needs to be made aware of the ARM library loader]

FAIL: OVERLOAD2 no expected error

FAIL: probefunc:.statement.(0xaddr).absolute shutdown (eof) [systemtap 
appears to be unable to translate the address back into the 
corresponding symbol.  Could be related to the debugpath failure?]

FAIL: systemtap.base/process_by_cmd.stp -c ./process_by_cmd [Oddly, 
systemtap is picking up a function return before the process starts. 
Possibly it's coming from libc?]

FAIL: sdt -O2  uprobe
FAIL: sdt c89  uprobe
FAIL: sdt c99  uprobe
FAIL: sdt c99 -pedantic uprobe
FAIL: sdt gnu99  uprobe
FAIL: sdt gnu99 -pedantic uprobe
FAIL: sdt c++98  uprobe
FAIL: sdt c++98 -pedantic uprobe
FAIL: sdt gnu++98  uprobe
FAIL: sdt gnu++98 -pedantic uprobe
FAIL: sdt c++0x  uprobe
FAIL: sdt c++0x -pedantic uprobe
FAIL: sdt gnu++0x  uprobe
FAIL: sdt gnu++0x -pedantic uprobe
[semantic error: unable to find local 'arg1' near pc 0x8408  in  call1 
/home/root/stuff/systemtap/testsuite/systemtap.base/sdt.c ( 
(alternatives: $a): identifier '$arg1' at 
/home/root/stuff/systemtap/testsuite/systemtap.base/sdt.stp:8:18.]

FAIL: sdt_va_args base
FAIL: sdt_va_args c89
FAIL: sdt_va_args c99
FAIL: sdt_va_args gnu99
FAIL: sdt_va_args c++98
FAIL: sdt_va_args gnu++98
FAIL: sdt_va_args c++0x
FAIL: sdt_va_args gnu++0x
[similar to the sdt failures above]

FAIL: compiling sdt.c c89 -pedantic uprobe 
[/home/root/stuff/systemtap/testsuite/systemtap.base/sdt.c:67:3: error: 
string length '518' is greater than the length '509' ISO C90 compilers 
are required to support]

ERROR: tcl error sourcing 
/home/root/stuff/systemtap/testsuite/systemtap.base/sdt_misc.exp [ERROR: 
child process exited abnormally]

FAIL: stmt_rel line numbers [semantic error: multiple addresses for...]

FAIL: 32_BIT_UTRACE_SYSCALL_ARGS startup (eof) [Warning: child process 
exited with signal 4 (Illegal instruction)]

FAIL: vta-test-m32-O
FAIL: vta-test-m32-O2
[semantic error: failed to retrieve location attribute for local 'a' 
(dieoffset: 0x181): identifier '$a' at 
/home/root/stuff/systemtap/testsuite/systemtap.base/vta-test.stp:2:27]

FAIL: dtrace_clone2 compilation
FAIL: dtrace_clone4 compilation
[WARNING: Can't parse SDT_V3 operand 'r3': identifier '$arg1' at 
/home/root/stuff/systemtap/testsuite/systemtap.clone/dtrace_clone.stp:6:47]

FAIL: dtrace_fork_exec2 compilation
FAIL: dtrace_fork_exec4 compilation
FAIL: dtrace_vfork_exec2 compilation
FAIL: dtrace_vfork_exec4 compilation
[Fail similarly to dtrace_clone]

FAIL: backtrace of yyy_func2 (0)
FAIL: print_stack of yyy_func2 (0)
FAIL: backtrace of yyy_func3 (0)
FAIL: print_stack of yyy_func3 (0)
FAIL: backtrace of yyy_func4 (0)
FAIL: print_stack of yyy_func4 (0)
FAIL: print_stack didn't find systemtap_test_module1 (0)
FAIL: print_stack didn't find [kernel] (0)
FAIL: function arguments: unexpected timeout
FAIL: all pid tests - unexpected EOF
FAIL: function arguments -- numeric: compilation failed
FAIL: function arguments -- numeric --kelf --ignore-dwarf: compilation 
failed
[semantic error: no match while resolving probe point 
module("systemtap-test-module2").function("yyy_int")]

FAIL: usymbols m32
FAIL: usymbols m32-O
FAIL: usymbols m32-O2
[line 2: expected "handler: lib_handler (.+/libusymbols-m32.so)", Got 
"handler: 0x400b450c (/home/root/stuff/test/libusymbols-m32.so)"]

FAIL: buildok/pretty.stp [Looks like I need to rerun this with the 
systemtap debug info installed]

FAIL: buildok/process_test.stp [semantic error: unable to find local 
'signr' near pc 0xc005f5b4  in  handle_signal...]

FAIL: buildok/scheduler-all-probes.stp [semantic error: no match while 
resolving probe point kernel.function("__switch_to")]

FAIL: buildok/seventeen.stp [semantic error: unable to find local 
'nfs_program' near pc 0xc01bc714  in  nfs_fsync_dir]

FAIL: buildok/syscalls-arch-detailed.stp
[semantic error: probe point mismatch at position 1]

FAIL: semok/config_number.stp [semantic error: probe point mismatch at 
position 0]

FAIL: semok/mangled.stp
FAIL: semok/pretty.stp
[similar to buildok/pretty.stp]

FAIL: semok/twentyseven.stp [semantic error: no match while resolving 
probe point module("no_such_module").function("no_such_function")]

FAIL: systemtap.stress/current.stp compilation [similar to 
buildok/process_test.stp]

FAIL: 32-bit access nd_syscall
FAIL: 32-bit acct nd_syscall
FAIL: 32-bit alarm nd_syscall
FAIL: 32-bit chmod nd_syscall
FAIL: 32-bit clock nd_syscall
FAIL: 32-bit dir nd_syscall
FAIL: 32-bit forkwait nd_syscall
FAIL: 32-bit futimes nd_syscall
FAIL: 32-bit itimer nd_syscall
FAIL: 32-bit link nd_syscall
FAIL: 32-bit mmap nd_syscall
FAIL: 32-bit mount nd_syscall
FAIL: 32-bit net1 nd_syscall
FAIL: 32-bit openclose nd_syscall
FAIL: 32-bit readwrite nd_syscall
FAIL: 32-bit rt_signal nd_syscall
FAIL: 32-bit select nd_syscall
FAIL: 32-bit sendfile nd_syscall
FAIL: 32-bit signal nd_syscall
FAIL: 32-bit stat nd_syscall
FAIL: 32-bit statfs nd_syscall
FAIL: 32-bit swap nd_syscall
FAIL: 32-bit sync nd_syscall
FAIL: 32-bit timer nd_syscall
FAIL: 32-bit trunc nd_syscall
FAIL: 32-bit uid nd_syscall
FAIL: 32-bit umask nd_syscall
FAIL: 32-bit unlink nd_syscall

FAIL: 32-bit alarm syscall
FAIL: 32-bit stat syscall

If anyone has any clues about the above failures, please let me know!

Thanks!

-Wade Farnsworth