patch for PR gdb/574

[david carlton <carlton at math dot stanford dot edu>]

For various reasons, I decided to delve into GDB recently, so I picked
bug 574 somewhat at random and decided to try to fix it.  Here's a
patch.

The problem: GDB tries to access an improper memory location while
trying to find run time type info on certain code compiled with G++
2.something.  In the situation in question, GDB is trying to find the
vptr of an object that is a subclass of a class with virtual functions
and static member data.  It looks for the information about the vptr
at a place where information about the static member data is actually
stored.  This is a data structure with a useless (and very large)
bitpos field; adding this bitpos to a pointer causes that pointer to
point to the middle of nowhere.  Oops.

The solution: Get rid of the check for

  if (VALUE_ENCLOSING_TYPE(v) != VALUE_TYPE(v))

and the surrounding code in gnu-v2-abi.c(gnuv2_value_rtti_type).

Some comments:

* gnu-v3-abi.c(gnuv3_rtti_type) doesn't have any such code, so I don't
  think this fix is likely to break anything that isn't also broken in
  gnu-v3-abi.c.

* It doesn't seem to cause any testsuites to fail that didn't fail
  before.  (In fact, when I ran it, it turned gdb.base/selftest.exp
  from a FAIL into a pass, for what that's worth; I don't know if my
  changes are relevant to that, though.)  I'm not too used to working
  the testsuites; please rerun them to make sure.

* In the test case in question, the only reason why that check matched
  was because a call to value_cast earlier in the function set the
  enclosing type to differ from the type.

* There are other problems in the surrounding code that are unrelated
  to this.  When I compiled my test program under g++-3.1, I got an
  unrelated bad memory access from inside gnu-v3-abi.c.  Also (and
  this may be related to that bad memory access), there seems to be a
  difference of opinion among various pieces of code as to whose job
  it is to demangle symbols that you want to look up in symbol
  tables.  I'll try to spend some time over the next week or two
  sorting this out.

* I'm not convinced that the code I deleted wouldn't be useful in some
  circumstances (where we had a pointer to something in the middle of
  a class); I'll try to spend some time over the next week or two
  coming up with test cases to see cause the code to fail, and to see
  if there's a better fix around somewhere.  Nonetheless, given that
  my patch does fix a known problem and, if there is a problem with
  enclosing_type, then the current gnu-v3-abi.c code probably has it
  as well, I think it should be committed.

* I haven't contacted the submitter of the bug report, but I'd be
  happy to do so if that is appropriate.

I'm including a sample program that demonstrates the bug (though the
submitter of the bug report also submitted a more elaborate test case
that has some other interesting features; hurrah for good bug
reports), a ChangeLog entry, and the patch; all will follow my
signature.

David Carlton
carlton@math.stanford.edu

/*
  An attempt to replicate GDB bug 574 with a shorter program.

  g++ -g test.cc  # no optimization, else we'd need to split into 2 files
  gdb a.out
  (gdb) b main
  (gdb) r
  (gdb) n
  (gdb) p *theB
*/

class A {
public:
  virtual void foo() {};		// Stick in a virtual function.
  static const int bar;			// Stick in a static data member.
};

class B : public A {
  static const int bar = 1;
};

int main()
{
  B *theB = new B;
}


2002-07-22  david carlton  <carlton@math.stanford.edu>

	* gnu-v2-abi.c (gnuv2_value_rtti_type): Eliminate test for being
	enclosed.  Fix PR gdb/574.


Index: gnu-v2-abi.c
===================================================================
RCS file: /cvs/src/src/gdb/gnu-v2-abi.c,v
retrieving revision 1.6
diff -c -p -r1.6 gnu-v2-abi.c
*** gnu-v2-abi.c	4 Jan 2002 18:20:19 -0000	1.6
--- gnu-v2-abi.c	22 Jul 2002 18:22:05 -0000
*************** gnuv2_value_rtti_type (struct value *v, 
*** 189,195 ****
    struct type *rtti_type;
    CORE_ADDR coreptr;
    struct value *vp;
-   int using_enclosing = 0;
    long top_offset = 0;
    char rtti_type_name[256];
    CORE_ADDR vtbl;
--- 189,194 ----
*************** gnuv2_value_rtti_type (struct value *v, 
*** 244,268 ****
    if (VALUE_ADDRESS (value_field (v, TYPE_VPTR_FIELDNO (known_type))) == 0)
      return NULL;
  
!   /*
!     If we are enclosed by something that isn't us, adjust the
!     address properly and set using_enclosing.
!   */
!   if (VALUE_ENCLOSING_TYPE(v) != VALUE_TYPE(v))
!     {
!       struct value *tempval;
!       int bitpos = TYPE_BASECLASS_BITPOS (known_type,
!                                           TYPE_VPTR_FIELDNO (known_type));
!       tempval=value_field (v, TYPE_VPTR_FIELDNO(known_type));
!       VALUE_ADDRESS(tempval) += bitpos / 8;
!       vtbl=value_as_address (tempval);
!       using_enclosing=1;
!     }
!   else
!     {
!       vtbl=value_as_address(value_field(v,TYPE_VPTR_FIELDNO(known_type)));
!       using_enclosing=0;
!     }
  
    /* Try to find a symbol that is the vtable */
    minsym=lookup_minimal_symbol_by_pc(vtbl);
--- 243,249 ----
    if (VALUE_ADDRESS (value_field (v, TYPE_VPTR_FIELDNO (known_type))) == 0)
      return NULL;
  
!   vtbl=value_as_address(value_field(v,TYPE_VPTR_FIELDNO(known_type)));
  
    /* Try to find a symbol that is the vtable */
    minsym=lookup_minimal_symbol_by_pc(vtbl);
*************** gnuv2_value_rtti_type (struct value *v, 
*** 304,311 ****
        if (full)
          *full=1;
      }
-   if (using_enc)
-     *using_enc=using_enclosing;
  
    return rtti_type;
  }
--- 285,290 ----