libffi doesn't keep the stack aligned to 16 bytes

[Neil Roberts <neil at linux dot intel dot com>]

Hi

GCC appears to assume by default that the stack is always aligned to 16
bytes when entering a function. Presumably this is to make it easier to
use the aligned access instructions to load data into SSE
registers. This means that if you have a variable with the aligned(16)
attribute then GCC knows that it doesn't have to add an extra
instruction to align the stack pointer because it will already be
aligned.

However, libffi doesn't appear to respect this. When it calls a function
it allocates space on the stack by decrementing the stack pointer
according to the amount of space required. This means the depending on
the number of arguments the stack may or may not be aligned.

This ends up causes crashes when calling libraries that use -mfpmath=sse
because then GCC will try to store temporary SSE variables on the stack
and it will assume it can use aligned access instructions.

Attached is a test case which demonstates the problem and a patch to fix
it. We are using this patch in MeeGo because all of the libraries are
compiled with SSE math so we were getting crashes with gjs.

Regards,
- Neil

#include <ffi.h>
#include <stdio.h>

/* This deliberately isn't static because otherwise GCC will inline in
   and then it will treat ptr&15 as a constant because it knows the
   alignment should 16 rather than actually looking at the address */
void
show_alignment (void *ptr)
{
  int address_mod_16 = ((unsigned int) ptr) & 15;

  printf ("%s: alignment offset is %i\n",
          address_mod_16 == 0 ? "GOOD" : "BAD ",
          address_mod_16);
}

static int
a_function (int arg)
{
  int an_aligned_local_variable __attribute__ ((aligned (16)));

  show_alignment (&an_aligned_local_variable);

  return 42;
}

int
main (int argc, char **argv)
{
  ffi_cif cif;
  ffi_type *args[] = { &ffi_type_sint };

  if (ffi_prep_cif (&cif, FFI_DEFAULT_ABI, 1, &ffi_type_sint, args) != FFI_OK)
    printf ("ffi_prep_cif failed\n");
  else
    {
      int argument = 3141592;
      void *argv[] = { &argument };
      int return_value;

      printf ("Calling function directly:\n");
      a_function (argument);

      printf ("Calling function via libffi:\n");
      ffi_call (&cif, (void (*) (void)) a_function, &return_value, argv);
    }

  return 0;
}

>From 89d2865bcf30bc020399a74b3053d46ef7750a28 Mon Sep 17 00:00:00 2001
From: Neil Roberts <neil@linux.intel.com>
Date: Fri, 26 Feb 2010 14:42:49 +0000
Subject: [PATCH] x86: Align the stack to 16-bytes before making the call

If gcc is targetting i686 then it will use SSE registers for the
floating point operations. It will then sometimes use 16-byte
temporary variables on the stack to store the registers and it will
use aligned instructions to access them. This will segfault if the
variable is not aligned to 16-bytes. Apparently GCC assumes that the
stack is 16-byte aligned when a function is entered and it uses this
to position its temporary variables. Therefore libffi needs to align
the stack or the called function will crash in some circumstances.
---
 src/x86/sysv.S |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/src/x86/sysv.S b/src/x86/sysv.S
index f4b6c1e..81374cc 100644
--- a/src/x86/sysv.S
+++ b/src/x86/sysv.S
@@ -48,6 +48,9 @@ ffi_call_SYSV:
 	movl  16(%ebp),%ecx
 	subl  %ecx,%esp
 
+        /* Align the stack pointer to 16-bytes */
+        andl  $0xfffffff0, %esp
+
 	movl  %esp,%eax
 
 	/* Place all of the ffi_prep_args in position  */
-- 
1.7.1.87.g94e70