Re: Update on freeze status of glibc 2.18?

["Carlos O'Donell" <carlos at redhat dot com>]

On 06/20/2013 08:40 PM, Torvald Riegel wrote:
> On Thu, 2013-06-20 at 16:09 -0700, Roland McGrath wrote:
>> What we'd do is change both PTHREAD_MUTEX_NORMAL and PTHREAD_MUTEX_DEFAULT
>> to be two different new values.  (Since PTHREAD_MUTEX_NORMAL is today
>> defined as PTHREAD_MUTEX_TIMED_NP, we'd have to change
>> PTHREAD_MUTEX_TIMED_NP as well.
> 
> Do you know of any source that defines the TIMED_NP semantics in detail?
> I haven't found anything clarifying whether they should behave like
> DEFAULT or like NORMAL.

Like NORMAL, but a timed mutex, which I assume means it can timeout
any of the standard locking functions even if not calling
pthread_mutex_timedlock.

Note that we have PTHREAD_MUTEX_TIMED_NP, PTHREAD_MUTEX_NORMAL, 
PTHREAD_MUTEX_DEFAULT, and PTHREAD_MUTEX_FAST_NP all with the same
value of 0.

There is no canonical source of documentation for any of the 
*_NP types in the current NPTL implementation (unless you consider
the kernel man pages project).

However the Linuxthreads implementation did document some of this
and NPTL aimed at being ABI and API compatible with Linuxthreads:
~~~
LinuxThreads supports only one mutex attribute: the mutex type, which is either PTHREAD_MUTEX_ADAPTIVE_NP for "fast" mutexes, PTHREAD_MUTEX_RECURSIVE_NP for "recursive" mutexes, PTHREAD_MUTEX_TIMED_NP for "timed" mutexes, or PTHREAD_MUTEX_ERRORCHECK_NP for "error checking" mutexes. As the NP suffix indicates, this is a non-portable extension to the POSIX standard and should not be employed in portable programs.

The mutex type determines what happens if a thread attempts to lock a mutex it already owns with pthread_mutex_lock. If the mutex is of the "fast" type, pthread_mutex_lock simply suspends the calling thread forever. If the mutex is of the "error checking" type, pthread_mutex_lock returns immediately with the error code EDEADLK. If the mutex is of the "recursive" type, the call to pthread_mutex_lock returns immediately with a success return code. The number of times the thread owning the mutex has locked it is recorded in the mutex. The owning thread must call pthread_mutex_unlock the same number of times before the mutex returns to the unlocked state.

The default mutex type is "timed", that is, PTHREAD_MUTEX_TIMED_NP.
~~~

Thus as you can surmise the FAST_NP behaves like NORMAL, and eveything
else which has the same value must behave like it also, and thus
deadlock on relock i.e. "suspends the calling thread forever."

>> Similar to the situation with DEFAULT, I
>> think we should make NORMAL be its own value distinct from TIMED_NP rather
>> than just changing them both to the same new value.  That way we'll have
>> the option in the future to give NORMAL some different implementation that
>> meets its POSIX requirements but need not match the particulars of TIMED_NP
>> if programs are written to request that specifically for some reason.)
>>
>> The old value (zero) would no longer be available by any public name, but
>> would be supported by the implementation as the COMPAT type.  Old binaries
>> requesting the COMPAT type could be given any new/future type that meets
>> the POSIX requirements for NORMAL.
>>
>> PTHREAD_MUTEX_INITIALIZER would be changed to use the new DEFAULT value.
>> Thus new binaries could be given any new/future type that meets the POSIX
>> requirements for DEFAULT.
> 
> That's a good idea, thanks.

What about TIMED_NP and FAST_NP? Also their own values? I think yes.

> What I don't understand in your plan is how we deal with the case that
> we have a PTHREAD_MUTEX_INITIALIZER in a program built against new glibc
> headers, but executed with an older glibc version.  In this case, the
> new type values coming from the initializer wouldn't be understood by
> the old pthread_mutex_lock(), for example.

Compiling against new headers and runtime, but running against
an old runtime has never been a supported scenario.

The assignment of the static initializer has no version information
and therefore can't be used to prevent the application from starting
up with the old runtime. Such a binary will fail with an assertion
check the first time it attempts to acquire a lock. That's the best
you can do if you change the static initializer value.

Therefore if we can avoid changing the static initializer value
that would be a better alternative.

>> pthread_mutexattr_gettype would also need a new version, so that its compat
>> function makes a pthread_mutexattr_t set to new-DEFAULT or new-NORMAL
>> report back as old-NORMAL.  But nothing else should need a new symbol version.
> 
> I need to think about this more, but could we keep the initializer at
> the old value (zero)?  We do not have an initializer for NORMAL, so
> there's no aliasing issue on the initializer side between NORMAL and
> DEFAULT; one needs to go through settype() to request NORMAL.
> 
> In settype(), we would translate from the old value for NORMAL (zero) to
> an internal new NORMAL_COMPAT type, and vice-versa in gettype().  This
> would allow us to treat initialized mutexes that still have the old
> value (zero) as type as the DEFAULT-typed mutexes that they are.  We
> would give DEFAULT a new type and not translate it, but it would have
> the same semantics as the zero-value type.
> 
> Thus, we would need new settype/gettype symbol versions, but just to be
> able to parse the new DEFAULT.  Old binaries would still use the old
> value for DEFAULT (zero) if using settype(), so they would get NORMAL
> semantics instead (which is fine) and couldn't use elision.  But in
> turn, old binaries that use just the initializers could make use of
> elision, because we disambiguate explicit requests for NORMAL via
> settype().
> 
> Does this sound reasonable?  Or are there any holes in this approach?

What prevents you from doing this *and* changing the initializer
value?

You have to accept that running a new application with an old runtime
won't give a good error message because you can't version the data
and you don't want to bump to SO. Maybe versioning something else
falsely might help, but runs afoul of what Roland was saying in that
it requires a new libpthread. I'm happy to call this scenario
unsupported and that the assertion is your runtime error.

Cheers,
Carlos.