Tricky issue with Python pretty-printer

[Peter Geoghegan <peter at 2ndquadrant dot com>]

Hello,

I have a user-level query about GDB - I hope that it's acceptable to
post such things to this list. It seemed that there was no more
appropriate list available.

I am using the Python GDB bindings to develop some pretty-printers for
PostgreSQL data-structures. My GDB version is:

GNU gdb (GDB) Fedora (7.3.50.20110722-13.fc16)
Copyright (C) 2011 Free Software Foundation, Inc.

Internally, PostgreSQL has a "node" infrastructure for representing
structures at all stages of query processing, from the earlier parsing
stage, through planning and query execution. It is not unlike the
Python PyObject infrastructure, in that there is a pseudo "abstract
base class", in Postgres's case "Node". Each node has a tag (an enum
field), and C's guarantees about struct field layout are used to X-ray
the type through a Node pointer and, potentially, cast and dereference
the node pointer to manipulate the underlying fields directly. Here is
the libpython implementation:

http://svn.python.org/projects/python/trunk/Tools/gdb/libpython.py

I would like to develop a pretty-printer that isn't overly cute and
clever, and yet does a few important things to save the user from
manually casting within the GDB command window. I am skeptical of the
idea of having my dispatcher function return a printer for pointer
types, so I've avoided that - pointers are just pointers. When Node
pointers are dereferenced within the command window, my pretty-printer
for the most part just does casting automatically, and nothing more.
This works well, up to a point.

I can get my code to do the right thing when a Node pointer is
dereferenced for most node types, because most node types only have an
enum field that corresponds in layout to the Node enum field (Node
consists entirely of an enum field - as I've said, it's a
super-class). There are, however, one or two important cases that I
can't seem to find a satisfactory way of handling. Some nodes have as
their first member a Scan struct. The Scan struct, in turn, has a Plan
struct as its first node. The Plan struct *does* have the same enum
field as its first field, so all of these structs can be manipulated
as nodes in the generic way. However, this plays havoc with my pretty
printer.

If I attempt to do the usual thing - to cast the value from within my
printer, before iterating through the casted-to value within my
printer's children() method - I get infinite recursion. This is
because I cannot differentiate between a dereferenced Plan pointer
(that is, a plan value) and the Plan field within the various plan
nodes when it comes time to cast. GDB is effectively trying to pretty
print a foo struct that contains, among other things, a foo struct.

I tried various grotty workarounds to attempt to fix the problem, to
no avail. It occurs to me that my approach may be less than idiomatic
- maybe I should get over my aversion to pretty-printing pointers
automatically (I do pretty print pointers, but only when recursively
printing a node tree, which I feel is substantively different). Any
insights would be greatly appreciated.


-- 
Peter Geoghegan       http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training and Services