This is the mail archive of the
cygwin
mailing list for the Cygwin project.
Re: Problem with multiprocessing module from Python
- From: Christopher Faylor <cgf-use-the-mailinglist-please at cygwin dot com>
- To: cygwin at cygwin dot com
- Date: Thu, 31 Oct 2013 22:59:11 -0400
- Subject: Re: Problem with multiprocessing module from Python
- Authentication-results: sourceware.org; auth=none
- References: <l4ei86$cos$2 at ger dot gmane dot org> <l4p2uq$ps8$1 at ger dot gmane dot org> <l4p6h3$801$1 at ger dot gmane dot org> <20131029205935 dot GC392 at ednor dot casa dot cgf dot cx> <l4p8oi$24u$1 at ger dot gmane dot org> <l4p8vj$24u$3 at ger dot gmane dot org> <20131030093313 dot GA28558 at calimero dot vinschen dot de> <l4u7er$2vj$1 at ger dot gmane dot org> <20131031184114 dot GD6599 at ednor dot casa dot cgf dot cx> <l4uflu$rrm$1 at ger dot gmane dot org>
- Reply-to: cygwin at cygwin dot com
On Thu, Oct 31, 2013 at 08:47:58PM +0000, Jean-Pierre Flori wrote:
>Le Thu, 31 Oct 2013 14:41:14 -0400, Christopher Faylor a ??crit??:
>
>> On Thu, Oct 31, 2013 at 06:27:39PM +0000, Jean-Pierre Flori wrote:
>>>Le Wed, 30 Oct 2013 10:33:13 +0100, Corinna Vinschen a ??crit??:
>>>
>>>> On Oct 29 21:22, Jean-Pierre Flori wrote:
>>>>> Le Tue, 29 Oct 2013 21:19:14 +0000, Jean-Pierre Flori a ??crit??:
>>>>>
>>>>> > Le Tue, 29 Oct 2013 16:59:35 -0400, Christopher Faylor a ??crit??:
>>>>> >> If you want this fixed, the easiest way to get that to happen is
>>>>> >> to post a simple test case which reproduces the problem. That is
>>>>> >> not the code snippet that you sent. A real working example would
>>>>> >> be required.
>>>>> > Sorry about that.
>>>>> >
>>>>> > Here you go:
>>>>> > """
>>>>> > from multiprocessing import Pool
>>>>> >
>>>>> > def f(x): return x
>>>>> >
>>>>> > p = pool(2)
>>>>> >
>>>>> > p.map(f, [1, 2])
>>>>> > """
>>>>> And I managed to introduce a typo. The third line should read Pool,
>>>>> so it is:
>>>>> """
>>>>> from multiprocessing import Pool
>>>>>
>>>>> def f(x): return x
>>>>>
>>>>> p = Pool(2)
>>>>>
>>>>> p.map(f, [1, 2])
>>>>> """
>>>>
>>>> Works for me. I guess. At least, if I run the script, nothing
>>>> happens:
>>>>
>>>> $ python x.py $
>>>>
>>>> Same on 32 and 64 bit Cygwin.
>>>>
>>>>
>>>> Corinna
>>>
>>>I think I got to the bottom of this.
>>>It seems the new implem of sem_getvalue in cgwin1.dll is the cause, see:
>>>http://cygwin.com/ml/cygwin-patches/2013-q3/msg00006.html It may also
>>>explain the random reproducibility if sval stays uninitialized or
>>>something like that (I did not check it is the case though).
>>
>> I doubt that was the problem. More likely it is something related to
>> the changes in thread.cc that followed that change.
>>
>> cgf
>
>I must admit that was just a guess.
>The thread.cc code changed whereas python code in Modules/
>_multiprocessing/semaphore.c did not.
>The python multiprocessing module (file semaphore.c) contains:
>"""
> if (sem_getvalue(self->handle, &sval) < 0) {
> return PyErr_SetFromErrno(PyExc_OSError);
> } else if (sval >= self->maxvalue) {
> PyErr_SetString(PyExc_ValueError, "semaphore or lock "
> "released too many times");
> return NULL;
> }
>"""
>Changing this to
>"""
> sval = sem_getvalue(self->handle, &sval);
> if (sem_getvalue(self->handle, &sval) < 0) {
> return PyErr_SetFromErrno(PyExc_OSError);
> } else if (sval >= self->maxvalue) {
> PyErr_SetString(PyExc_ValueError, "semaphore or lock "
> "released too many times");
> return NULL;
> }
>"""
>to emulate the previous behavior of sem_getvalue seems to solve my problem
>(and was easier than rebuilding cygwin1.dll).
That doesn't emulate the previous behavior. The return value of sem_getvalue
was changed to 0 or -1 as per POSIX. If self->maxvalue is > 1 then
you won't see an error but it won't be correct either.
cgf
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple