This is the mail archive of the cygwin mailing list for the Cygwin project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: Problem with multiprocessing module from Python


On Thu, Oct 31, 2013 at 08:47:58PM +0000, Jean-Pierre Flori wrote:
>Le Thu, 31 Oct 2013 14:41:14 -0400, Christopher Faylor a ??crit??:
>
>> On Thu, Oct 31, 2013 at 06:27:39PM +0000, Jean-Pierre Flori wrote:
>>>Le Wed, 30 Oct 2013 10:33:13 +0100, Corinna Vinschen a ??crit??:
>>>
>>>> On Oct 29 21:22, Jean-Pierre Flori wrote:
>>>>> Le Tue, 29 Oct 2013 21:19:14 +0000, Jean-Pierre Flori a ??crit??:
>>>>> 
>>>>> > Le Tue, 29 Oct 2013 16:59:35 -0400, Christopher Faylor a ??crit??:
>>>>> >> If you want this fixed, the easiest way to get that to happen is
>>>>> >> to post a simple test case which reproduces the problem.  That is
>>>>> >> not the code snippet that you sent.  A real working example would
>>>>> >> be required.
>>>>> > Sorry about that.
>>>>> > 
>>>>> > Here you go:
>>>>> > """
>>>>> > from multiprocessing import Pool
>>>>> > 
>>>>> > def f(x): return x
>>>>> > 
>>>>> > p = pool(2)
>>>>> > 
>>>>> > p.map(f, [1, 2])
>>>>> > """
>>>>> And I managed to introduce a typo. The third line should read Pool,
>>>>> so it is:
>>>>> """
>>>>> from multiprocessing import Pool
>>>>> 
>>>>> def f(x): return x
>>>>> 
>>>>> p = Pool(2)
>>>>> 
>>>>> p.map(f, [1, 2])
>>>>> """
>>>> 
>>>> Works for me.  I guess.  At least, if I run the script, nothing
>>>> happens:
>>>> 
>>>>   $ python x.py $
>>>> 
>>>> Same on 32 and 64 bit Cygwin.
>>>> 
>>>> 
>>>> Corinna
>>>
>>>I think I got to the bottom of this.
>>>It seems the new implem of sem_getvalue in cgwin1.dll is the cause, see:
>>>http://cygwin.com/ml/cygwin-patches/2013-q3/msg00006.html It may also
>>>explain the random reproducibility if sval stays uninitialized or
>>>something like that (I did not check it is the case though).
>> 
>> I doubt that was the problem.  More likely it is something related to
>> the changes in thread.cc that followed that change.
>> 
>> cgf
>
>I must admit that was just a guess.
>The thread.cc code changed whereas python code in Modules/
>_multiprocessing/semaphore.c did not.
>The python multiprocessing module (file semaphore.c) contains:
>"""
>        if (sem_getvalue(self->handle, &sval) < 0) {
>            return PyErr_SetFromErrno(PyExc_OSError);
>        } else if (sval >= self->maxvalue) {
>            PyErr_SetString(PyExc_ValueError, "semaphore or lock "
>                            "released too many times");
>            return NULL;
>        }
>"""
>Changing this to
>"""
>        sval = sem_getvalue(self->handle, &sval);
>        if (sem_getvalue(self->handle, &sval) < 0) {
>            return PyErr_SetFromErrno(PyExc_OSError);
>        } else if (sval >= self->maxvalue) {
>            PyErr_SetString(PyExc_ValueError, "semaphore or lock "
>                            "released too many times");
>            return NULL;
>        }
>"""
>to emulate the previous behavior of sem_getvalue seems to solve my problem 
>(and was easier than rebuilding cygwin1.dll).

That doesn't emulate the previous behavior.  The return value of sem_getvalue
was changed to 0 or -1 as per POSIX.  If self->maxvalue is > 1 then
you won't see an error but it won't be correct either.

cgf

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]