Re: eliminate deprecated_insert_raw_breakpoint. what's left.

["Ulrich Weigand" <uweigand at de dot ibm dot com>]

Peter Schauer wrote:

> I hope to be able to shed some light on this problem, although it
> is more than fifteen years ago that I did some work for GDB on AIX.
> 
> From my notes back then, AIX 3 and AIX 4 had a very peculiar ptrace
> implementation, where the current ptrace state of the inferior process
> (including the current process registers) was maintained approximately
> 512 bytes below the current user stack pointer of the process.
> 
> This resulted in problems with AIX inferior function calls.
> If the called function takes one or more large aggregate parameters
> by value, or if you pass a large amount of parameters, the ptrace
> area gets corrupted, when the dummy function call parameters are
> pushed on the user stack, due to this awkward AIX stack layout.

Thanks for providing this background!

> To work around this problem, the execution of a dummy instruction
> (when altering the stack pointer) caused the kernel to move the ptrace
> state area further below on the user stack, allowing GDB to write below
> the current user stack safely.
> In GDB 6.x, rs6000_push_dummy_call even secured the stack partially during
> pushing of the arguments, via an additional call of
> regcache_raw_write_signed to gdbarch_sp_regnum (gdbarch), which is
> no longer present in current versions of GDB.

Well, I still see this:
  /* Set the stack pointer.  According to the ABI, the SP is meant to
     be set _before_ the corresponding stack space is used.  On AIX,
     this even applies when the target has been completely stopped!
     Not doing this can lead to conflicts with the kernel which thinks
     that it still has control over this not-yet-allocated stack
     region.  */
  regcache_raw_write_signed (regcache, gdbarch_sp_regnum (gdbarch), sp);

and:
      /* This is another instance we need to be concerned about
         securing our stack space.  If we write anything underneath %sp
         (r1), we might conflict with the kernel who thinks he is free
         to use this area.  So, update %sp first before doing anything
         else.  */

      regcache_raw_write_signed (regcache,
                                 gdbarch_sp_regnum (gdbarch), sp);

Are there other instances where this is missing?

> Executing the dummy instruction is very fragile, especially if signals
> get involved during the execution, and it didn't even help, if more
> than ~100 bytes of parameters were pushed on the user stack on AIX 4.
> Back then, there was no other choice though.
> 
> Unfortunately I do not know, if this peculiar AIX stack layout is still
> used in AIX 5 or later, maybe Ulrich Weigand could tell you more about it.

I don't know off-hand.  I'll try to find out.
 
> I think you could/should zap exec_one_dummy_insn, provided that you test
> a dummy function call on the oldest AIX version that GDB has to support,
> with a large aggregate parameter, which is passed by value.

The only version I have ready access to is AIX 7.1, and on this there
are no testsuite regression (and in fact, quite a number of failures
seem to go away!) when zapping exec_one_dummy_insn.

I'm not sure which versions we need to / should support in GDB; I guess
the oldest version where the OS itself is still supported by IBM is 6.1.

Bye,
Ulrich

-- 
  Dr. Ulrich Weigand
  GNU/Linux compilers and toolchain
  Ulrich.Weigand@de.ibm.com