Re: [PATCH][gdb] fix unsigned overflow in charset.c

[Pedro Alves <palves at redhat dot com>]

On 10/09/2018 08:58 PM, John Baldwin wrote:
> On 10/9/18 11:10 AM, Paul Koning wrote:
>>
>>
>>> On Oct 9, 2018, at 1:57 PM, John Baldwin <jhb@FreeBSD.org> wrote:
>>>
>>> On 10/9/18 10:40 AM, Paul Koning wrote:
>>>>
>>>>
>>>>> On Oct 9, 2018, at 1:31 PM, Pedro Alves <palves@redhat.com> wrote:
>>>>>

>>> I also ran into the same failure using LLVM's ubsan on FreeBSD but in a different
>>> use of obstack_blank_fast().  If we wanted to fix this, I wonder if we'd instead
>>> want to fix it centrally in obstack_blank_fast (e.g. by using a ptrdiff_t cast)
>>> rather than fixing various consumers of the API.  That would be a change to
>>> libiberty though, not just gdb.
>>
>> I suppose.  But casts in macros scare me, they can hide mistakes.  It seems more reasonable to have the caller be responsible for creating a value of the correct type.  Since it's an adjustment, I suppose the cast should be for ptrdiff_t rather than ssize_t?
> 
> So if obstack_blank_fast() were an inline function instead of a macro, I
> suspect it's second argument would be of type ptrdiff_t in which case the
> equivalent "hidden" cast would happen at the function call.  That said,
> the obstack_blank() macro uses _OBSTACK_SIZE_T (which is an unsigned size_t)
> when it declares a local variable to pass as the offset, so it seems obstack
> really is relying on unsigned wrap around.

The function is documented to take an int, at least:

 void obstack_blank_fast (struct obstack *obstack-ptr, int size)

 https://www.gnu.org/software/libc/manual/html_node/Summary-of-Obstacks.html

Looks like some of the "int"-ness was lost with the obstack v2 changes
a while ago, to support larger (64-bit) objects.

If I diff my system's obstack.h with libiberty's local copy, I see:

(This is Fedora 27, a little outdated wrt to glibc in use by now.
 Upstream glibc's obstack.h is in sync with liberty's IIRC.)

$ diff -upw /usr/include/obstack.h obstack.h | less

-#ifdef __PTRDIFF_TYPE__
-# define PTR_INT_TYPE __PTRDIFF_TYPE__
+#if _OBSTACK_INTERFACE_VERSION == 1
+/* For binary compatibility with obstack version 1, which used "int"
+   and "long" for these two types.  */
+# define _OBSTACK_SIZE_T unsigned int
+# define _CHUNK_SIZE_T unsigned long
+# define _OBSTACK_CAST(type, expr) ((type) (expr))
 #else
-# include <stddef.h>
-# define PTR_INT_TYPE ptrdiff_t
+/* Version 2 with sane types, especially for 64-bit hosts.  */
+# define _OBSTACK_SIZE_T size_t
+# define _CHUNK_SIZE_T size_t
+# define _OBSTACK_CAST(type, expr) (expr)
 #endif


and 

@@ -359,11 +375,10 @@ extern int obstack_exit_failure;
 # define obstack_blank(OBSTACK, length)                                              \
   __extension__                                                                      \
     ({ struct obstack *__o = (OBSTACK);                                              \
-       int __len = (length);                                                 \
-       if (__o->chunk_limit - __o->next_free < __len)                        \
+       _OBSTACK_SIZE_T __len = (length);                                     \
+       if (obstack_room (__o) < __len)                                       \
         _obstack_newchunk (__o, __len);                                      \
-       obstack_blank_fast (__o, __len);                                              \
-       (void) 0; })
+       obstack_blank_fast (__o, __len); })


Note how above we used to have "int __len = (length);"

But that's obstack_blank, not obstack_blank_fast.  The latter
never had a cast.

Not sure what's best to do, but I think I leaning toward
agreeing with Paul, in that passing down a signed negative
integer rather than passing down a large unsigned integer
expecting it to cast to a negative integer ends up
being a little better.

Thanks,
Pedro Alves