Re: Python API plans

[Phil Muldoon <pmuldoon at redhat dot com>]

On 24/08/10 02:06, Tom Tromey wrote:
> Phil asked me to write up a list of the various Python projects that I
> think we should pursue in gdb.  So, here it is; I welcome feedback,
> additions, etc.  Don't expect many details here, it is really just the
> result of making notes whenever I missed something over the last couple
> of years.
> 
> * Rebase and polish Oguz's inferior event work from the 2009 SoC.  I
>   think Oguz has started looking into this again.  This functionality
>   blocks a number of interesting Python uses; I think the event registry
>   work is needed for some other (smaller) projects too.

There are two areas I would like to see improved beyond the initial
patch.  I have no opinion whether these can be incrementally improved
after submission, or if we should hack on them first.

* Events should report rich data when triggered.  So when a
  breakpoint event occurs it should also have an object  payload
  describing the breakpoint event. (IE, a gdb.Breakpoint object that
  describes the breakpoint that was triggered).  Each event should have
  this object payload.

* Event classes should be able to be written in Python as well as
  C. Hopefully this would eliminate some boilerplate code and make it
  more accessible to the Python developer.  This probably means
  writing an API wrapper around the various observer-* routines that
  GDB already exports for various inferior events.  Also, the existing
  observer patterns may not offer full coverage to the amount of
  events we can respond to in Python, so we might need to add more.
  Or figure/adapt how MI reports on various inferior events.

> 
> * Fix all open bugs.
> 
> * Polish some of the existing commands and functions from the
>   archer-tromey-python branch.  Sub-tasks:
> 
>   * Finish the command-loading code.  Time eager loading to see how
>     expensive it is; if too expensive, write some lazy-loading
>     wrappers.  (We definitely do not want the "require" command from the
>     branch.)
> 
>   * (See below for info on frame wrapper, new-backtrace, etc.)


I agree about the eager loading.  It might amount to nothing at all.
If it does turn out to be expensive, what about having commands
register themselves against a global Python list? This would be
similar to how pretty-printers work.  We could defer loading until the
command is needed, but still (with some alteration to the completion
code) allow the user to see the command-list in 'help 'and make the
commands available for tab-completion?


> * Write Python wrappers for all MI commands, see:
> 
>     http://sourceware.org/bugzilla/show_bug.cgi?id=11688
> 
>   This is an easy (-ish) way to provide a simple API to most of the
>   relevant parts of GDB.

Andre picked up my thoughts here, so I won't embellish.

> 
> * Generic command improvements.
> 
>   * Sometimes it would be nice to write a command with the same name as
>     an existing command, but then delegate to the existing command (say,
>     via a super call).

Do you mean purely Python commands overriding other Python commands, or
in-built commands as well?
 
>   * Likewise it would be nice if you could easily uninstall an
>     overriding command.


Or any command after loading.

 
> * Breakpoint improvements:
> 
>   * Make it possible to create an internal breakpoint from Python.
> 
>   * Add a bit to breakpoints to make the breakpoint "maybe non-stopping"
>     -- in the sense that when the breakpoint is hit, it will call an
>     attached Python function, and if the function returns True, the
>     breakpoint will not stop.  This would be different from "commands"
>     including "cont" in that it would not interfere with step or next.
>     (You can fake this right now by doing work via a convenience
>     function attached to the breakpoint condition -- but that is gross
>     and user-visible.)


Well you can already make a Python breakpoint conditional, so you
could just predicate the breakpoint stopping the inferior by setting the
appropriate condition..  I'm not sure if you can run arbitrary
Python code in a condition though (i.e., .condition("python if
somePythonFunction() == True").  It would be nice to do it more
formally anyway (as you note, I think we are violently agreeing ;).  


>   * Use these two new features to write a command to create a "return"
>     breakpoint.  This is a breakpoint that fires when a function
>     returns.  (I think you can do this by putting a breakpoint at the
>     function's entry, then going up and looking at the outer frame's
>     PC, or something along those lines.  Bonus points for interacting
>     properly with longjmp and throw...)

Nice!


> * Minimal systemtap integration.  In particular I've been thinking it
>   would be nice if gdb could read and use systemtap probe points.  This
>   doesn't have to be written in Python, but it seems like a nice
>   experiment.  Some sub-tasks:
> 
>   * Hooking into linespecs so users can write "probe:name" to refer to a
>     systemtap probe.  This would also mean dealing with completion,
>     maybe some ugly linespec refactoring.
> 
>   * Let Python code access the contents of a section in an objfile.


It would be nice to be able to set mnemonic breakpoints on System Tap probe
points as well.  And somehow export the probe-points via debug-info (or however) so
the user can tab-complete them (and make all of this available through
the Python API as well).


> * Make a new Expression object that represents a parsed expression.
>   Add a new gdb.parse method to create these, something like:
> 
>     gdb.parse (EXPR, [BLOCK], [STOP_AT_COMMA]) -> gdb.Expression
> 
>   This functionality is handy for writing certain kinds of commands.
> 
> * Frame and "new-backtrace" cleanup
> 
>   * Clean up the frame wrapper and iteration code to solve problems
>     reported on the Archer list; look at the various use cases posted
>     there to make sure they can all be handled.  This may mean adding
>     more methods to Frame so we can avoid having a FrameWrapper at all.
> 
>   * Add a feature to new-backtrace to print the objfile name for a
>     frame.  (This might be a request in bugzilla somewhere.)
> 
>   * Rewrite the "upto" family of commands.  They are ok but there were
>     some ideas on the Archer list about how to improve them.
> 
>   * Give frame wrappers, or their replacement, some notion of being
>     synthetic and having "original" sub-frames.  The idea here is to
>     provide multiple views of the stack, as a start toward debugging
>     scripting languages.  Provide versions of up/down/etc that work on
>     the filtered view or the raw view, enhance new-backtrace to display
>     sub-frames indented or something like that.  Figure out how to
>     present this via MI.
> 

These are all good suggestions, thanks.

Ideally I'd like the Python API to be fully aware of the state of the
inferior. On the flip-side I'd also to be able to control the
inferior in a more atomic way than issuing gdb.execute("step")
commands and guessing if the inferior did as expected.

There are few small others things. Some of these are in the wiki.

* Python breakpoints to support 'catch' breakpoints (especially the
  system-call variety).

* Make it so that tab completion knows about Python API
  completions. So (gdb) gdb.<TAB> would list all of the available
  Python bits associated with the GDB module.

* Python representation of a target.

* Allow Python scripts to control start-up of GDB so it allows the
  equivalent of gcore like commands.

I talked with Oguz about the event inferior code.  I'll write up the
conclusion to that email soon.

Cheers

Phil