Re: [RFC] [PATCH] Provide the ability to write the frame unwinder in Python

[Alexander Smundak <asmundak at google dot com>]

Ping.

On Thu, Feb 12, 2015 at 9:58 AM, Alexander Smundak <asmundak@google.com> wrote:
> On Wed, Feb 4, 2015 at 2:35 PM, Doug Evans <dje@google.com> wrote:
>> High level comments:
>>
>> Is it possible to see the code, and example usage, of a real-life use-case
>> of this? That will help folks not familiar with this project to understand
>> the problem we are trying to solve.
> The case in question is Java Virtual Machine. It is a JIT compiler for Java,
> and it is part of the OpenJDK (Java Development Kit). It compiles Java
> methods on the fly, and the emitted code omits frame pointers (e.g., on
> x86_64 platform RBP is used as a general purpose register rather than
> as frame pointer). And, the emitted code does not have unwind info
> expected by GDB, so standard sniffers fail and the traceback stops when
> it encounters a frame for the JIT-compiled code.
> If we know how JVM works, we know where to find the descriptors of the
> currently compiled code, and once we locate the descriptor for a given PC,
> we can extract the frame size, and unwind the frame.
> It's easier to have a custom sniffer than to make JVM maintain DWARF
> unwind info. Besides, most of the code for the sniffer is reused by the
> corresponding frame decorator.
> The full implementation of the combined sniffer/frame filter for OpenJDK
> is about 2500 lines and will eventually become part of it. I am waiting for
> this GDB patch to be reviewed before I can present it to be reviewed by
> the JDK community :-)
>
>> I'm still not sure what kind of performance cost we're looking at here as
>> it scales up, I can imagine most times there'll be no Python sniffers,
>> or at most one or two.  But it would be good to collect some perf data
>> (e.g., install 1,10,100 no-op sniffers and see if there's any measurable
>> difference in backtrace performance).
> I ran a test that walks a 100-frame stack calling 100 Python sniffers
> per frame, and it executes (throught the dejagnu checker) in 500ms on
> Xeon E5-1650 0 @ 3.20GHz.
> Please let me know if you would like it to be added to gdb/testsuite.
>
>> Exposing frame id implementation details (sp,pc,special), and the
>> form of how to do that, is something the community needs to decide on.
>> I think we can come up with something suitable, though perhaps not
>> the current form.
>
> The revised patch is attached. The important differences are as follows:
> * A sniffer is now an object, using the pattern similar to xmethods
> and pretty printers.
> * Register values are properly types (that is, based on a register type)
>
> I am still not certain whether it's worth changing SnifferInfo.read_register to
> have registers retrieved by name rather than by its number. Perhaps
> adding gdb.Architecture.register_name_to_number method will be
> a reasonable tradeoff?
>
> The documentation is obviously unfinished (to be done once the design
> issues are resolved), and what exists needs to be put into proper
> English.
>
> Here's take two:
>
> gdb/ChangeLog:
> 2015-02-28  Sasha Smundak  <asmundak@google.com>
>
>         * Makefile.in (SUBDIR_PYTHON_OBJS): Add py-unwind.o.
>         (SUBDIR_PYTHON_SRCS): Add py-unwind.c.
>         (py-unwind.o): New recipe.
>         * NEWS: mention Python frame unwinding.
>         * data-directory/Makefile.in (PYTHON_FILE_LIST):  Add sniffers.py.
>         * doc/python.texi (Writing a Frame Unwinder in Python): Add
>         section.
>         * python/lib/gdb/__init__.py (packages): Add frame_sniffers list.
>         * python/lib/gdb/command/sniffers.py: New file, implements GDB
>         commands to list/enable/disable Python sniffers.
>         * python/lib/gdb/function/sniffers.py: New file, implements
>         execute_sniffers function.
>         * python/lib/gdb/sniffer.py: New file, contains Sniffer class and
>         register_sniffer function.
>         * python/py-objfile.c (objfile_object): Add frame_sniffers field.
>         (objfpy_dealloc): Decrement frame_sniffers reference count.
>         (objfpy_initialize): Create frame_sniffers list.
>         (objfpy_get_frame_sniffers): Implement Objfile.frame_sniffers
>         getter.
>         (objfpy_set_frame_sniffers): Implement Objfile.frame_sniffers
>         setter.
>         (objfile_getset): Add frame_sniffers attribute to Objfile.
>         * python/py-progspace.c (pspace_object): Add frame_sniffers field.
>         (pspy_dealloc): Decrement frame_sniffers reference count.
>         (pspy_initialize): Create frame_sniffers list.
>         (pspy_get_frame_sniffers): Implement gdb.Progspace.frame_sniffers
>         getter.
>         (pspy_set_frame_sniffers): Implement gdb.Progspace.frame_sniffers
>         setter.
>         (pspy_getset): Add frame_sniffers attribute to gdb.Progspace.
>         * python/py-unwind.c: New file, implements Python frame sniffers
>         interface.
>         * python/python-internal.h (pspy_get_name_sniffers): New prototype.
>         (objpy_get_frame_sniffers): New prototype.
>         (gdbpy_initialize_unwind): New prototype.
>         * python/python.c (gdbpy_apply_type_printers): Call
>         gdbpy_initialize_unwind.
>
> gdb/testsuite/ChangeLog:
> 2014-02-30  Sasha Smundak  <asmundak@google.com>
>
>         * gdb.python/py-unwind-maint.c: Test program for py-unwind-maint.
>         * gdb.python/py-unwind-maint.exp: Tests sniffer-related GDB
>         commands.
>         * gdb.python/py-unwind-maint.py: Pythons sniffers for the test.
>         * gdb.python/py-unwind.c: Test program for the py-unwind test.
>         * gdb.python/py-unwind.exp: Python frame sniffers test.
>         * gdb.python/py-unwind.py: Frame sniffer in Python tested by
>         py-unwind test.