Skip to content

Locks broken wrt timeouts on Windows #55827

Closed

Description

@sbt
mannequin
BPO 11618
Nosy @loewis, @pitrou, @kristjanvalur, @tjguk, @briancurtin
Files
  • test-timeout.py
  • locktimeout.patch
  • semlocknt.patch
  • locktimeout2.patch
  • critlocknt.patch
  • locktimeout3.patch
  • ntlocks.patch
  • ntlocks.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2012-06-07.10:58:19.061>
    created_at = <Date 2011-03-20.17:06:57.180>
    labels = ['type-bug']
    title = 'Locks broken wrt timeouts on Windows'
    updated_at = <Date 2012-06-07.10:58:19.060>
    user = 'https://bugs.python.org/sbt'

    bugs.python.org fields:

    activity = <Date 2012-06-07.10:58:19.060>
    actor = 'kristjan.jonsson'
    assignee = 'none'
    closed = True
    closed_date = <Date 2012-06-07.10:58:19.061>
    closer = 'kristjan.jonsson'
    components = []
    creation = <Date 2011-03-20.17:06:57.180>
    creator = 'sbt'
    dependencies = []
    files = ['21304', '21306', '21308', '21322', '21325', '21336', '25271', '25351']
    hgrepos = []
    issue_num = 11618
    keywords = ['patch']
    message_count = 54.0
    messages = ['131515', '131520', '131521', '131524', '131525', '131527', '131529', '131530', '131533', '131534', '131635', '131636', '131642', '131643', '131650', '131656', '131661', '131669', '131670', '131672', '131676', '131682', '131684', '131685', '131686', '131695', '131699', '131702', '131728', '131732', '131744', '131745', '132618', '132619', '158713', '158720', '158830', '159071', '159072', '159081', '159210', '159427', '159429', '159432', '159447', '159459', '159465', '159466', '159468', '159681', '159686', '159819', '159949', '162468']
    nosy_count = 7.0
    nosy_names = ['loewis', 'pitrou', 'kristjan.jonsson', 'tim.golden', 'brian.curtin', 'python-dev', 'sbt']
    pr_nums = []
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue11618'
    versions = ['Python 3.2', 'Python 3.3']

    Activity

    1. sbt commented on Mar 20, 2011

      sbtmannequin
      MannequinAuthor

      In thread_nt.h, when the WaitForSingleObject() call in
      EnterNonRecursiveMutex() fails with WAIT_TIMEOUT (or WAIT_FAILED) the
      mutex is left in an inconsistent state.

      Note that the first line of EnterNonRecursiveMutex() is the comment

      /* Assume that the thread waits successfully */
      

      Allowing EnterNonRecursiveMutex() to fail with a timeout obviously
      violates this promise ;-) I think the problem was introduced to Python
      3.2 with:

       bpo-7316: Add a timeout functionality to common locking operations.
      

      The following Windows session demonstrates unexpected behaviour:

      Python 3.3a0 (default, Mar 19 2011, 18:16:48) [MSC v.1500 32 bit (Intel)] on win32
      Type "help", "copyright", "credits" or "license" for more information.
      >>> import threading
      >>> l = threading.Lock()
      >>> l.acquire()
      True
      >>> l.acquire(timeout=1)
      False
      >>> l.release()
      >>> l.locked()                  # should return False
      True
      >>> l.acquire(blocking=False)   # should return True
      False

      Also, after a timeout, uncontended acquires/releases always take the
      slow path:

      D:\Repos\cpython\PCbuild>python -m timeit ^
      More? -s "from threading import Lock; l = Lock()" ^
      More? "l.acquire();l.release()"
      1000000 loops, best of 3: 0.974 usec per loop

      D:\Repos\cpython\PCbuild>python -m timeit ^
      More? -s "from threading import Lock; l = Lock()" ^
      More? -s "l.acquire();l.acquire(timeout=0.1);l.release()" ^
      More? "l.acquire();l.release()"
      100000 loops, best of 3: 2.18 usec per loop

      A unit test is attached which passes on Linux but has three failures
      on Windows.

      The "owned" field of NRMUTEX is a count of the number of threads
      waiting for the mutex (not including the owner). "owned" will
      over-estimate the number of waiters if a timeout occurs, because the
      timed out thread will still be counted as a waiter.

      The obvious fix is to decrement mutex->owned when a timeout occurs.
      Unfortunately that would introduce a race which might allow two
      threads to think they own the lock at the same time.

      I also notice that EnterNonRecursiveMutex() wrongly sets
      mutex->thread_id to the current thread even when it fails with a
      timeout. It appears that the thread_id field is never actually used
      -- is it there to help with debugging? Perhaps it should just be
      removed.

      BTW only thread_pthread.h and thread_nt.h have implementations of
      PyThread_acquire_lock_timed(). Since this function appears to be
      required by _threadmodule.c, does this mean that in Python 3.2
      threads are only supported with pthreads and win32? If so you can
      get rid of all those other thread_*.h files.

    2. added
      type-bugAn unexpected behavior, bug, or error
      on Mar 20, 2011
    3. pitrou commented on Mar 20, 2011

      @pitrou
      Member

      It appears that the thread_id field is never actually used
      -- is it there to help with debugging? Perhaps it should just be
      removed.

      True, I think we can remove it.

      does this mean that in Python 3.2
      threads are only supported with pthreads and win32? If so you can
      get rid of all those other thread_*.h files.

      Getting ridding them is scheduled for 3.3.

    4. sbt commented on Mar 20, 2011

      sbtmannequin
      MannequinAuthor

      First stab at a fix.

      Gets rid of mutex->thread_id and adds a mutex->timeouts counter.

      Does not try to prevent mutex->owned from overflowing.

      When no timeouts have occurred I don't think it changes behaviour, and it uses the same number of Interlocked functions.

    5. pitrou commented on Mar 20, 2011

      @pitrou
      Member

      Well, Windows 2000 has semaphores, so why not use them? It makes the code much simpler. Patch attached (including test).

    6. sbt commented on Mar 20, 2011

      sbtmannequin
      MannequinAuthor

      Have you tried benchmarking it?

      Interlocked functions are *much* faster than Win32 mutex/semaphores in the uncontended case.

      It only doubles the time taken for a "l.acquire(); l.release()" loop in Python code, but at the C level it is probably 10 times slower.

      Do you really want the GIL to be 10 times slower in the uncontended case? ;-)

    7. pitrou commented on Mar 20, 2011

      @pitrou
      Member

      Have you tried benchmarking it?

      Interlocked functions are *much* faster than Win32 mutex/semaphores in
      the uncontended case.

      Well, I'd rather have obviously correct code than
      difficult-to-understand "speedy" code.

      The patch I've posted takes less than a microsecond per acquire/release
      pair, and that's in a virtual machine to begin with.

      Do you really want the GIL to be 10 times slower in the uncontended case? ;-)

      The GIL doesn't use these functions (see ceval_gil.h).

    8. loewis commented on Mar 20, 2011

      loewismannequin
      Mannequin

      Interestingly, it used to be a Semaphore up to [5e6e9e893acd]; in [cde4da18c4fa], Yakov Markovitch rewrote this to be the faster implementation we have today.

    9. pitrou commented on Mar 20, 2011

      @pitrou
      Member

      Interestingly, it used to be a Semaphore up to [5e6e9e893acd]; in
      [cde4da18c4fa], Yakov Markovitch rewrote this to be the faster
      implementation we have today.

      At that time, the Pythread_* functions were still in use by the GIL
      implementation, and it made a difference judging by the commit message.

    10. loewis commented on Mar 20, 2011

      loewismannequin
      Mannequin

      At that time, the Pythread_* functions were still in use by the GIL
      implementation, and it made a difference judging by the commit message.

      Hmm. And if some application uses thread.lock heavily, won't it still
      make a difference?

    11. pitrou commented on Mar 20, 2011

      @pitrou
      Member

      > At that time, the Pythread_* functions were still in use by the GIL
      > implementation, and it made a difference judging by the commit message.

      Hmm. And if some application uses thread.lock heavily, won't it still
      make a difference?

      An acquire/release pair is less than one microsecond here. Compared to
      the evaluation overhead of Python code, it seems not very significant.
      That said, if someone can guarantee than the complex approach is
      correct, why not.

    12. kristjanvalur commented on Mar 21, 2011

      kristjanvalurmannequin
      Mannequin

      Yes, the race condition with the timeout is a problem.
      Here is a patch that implements this lock using a condition variable.
      I agree that one must consider performance/simplicity when doing this.

    13. pitrou commented on Mar 21, 2011

      @pitrou
      Member

      Yes, the race condition with the timeout is a problem.
      Here is a patch that implements this lock using a condition variable.
      I agree that one must consider performance/simplicity when doing this.

      I don't understand why you need something that complicated. A simple
      semaphore should be enough (as in the POSIX implementation).

    14. kristjanvalur commented on Mar 21, 2011

      kristjanvalurmannequin
      Mannequin

      I'm just providing this as a fast alternative to the Semaphore, which as far as I know, will cause a kernel call every time.

      Complicated is relative. In terms of the condition variable api, I wouldn't say that it is. But given the fact that we have to emulate condition variables on older windows, then yes, it is complex.

      If we are rolling our own instead of using Semaphores (as has been suggested for performance reasons) then using a Condition variable is IMHO safer than a custom solution because the correctness of that approach is so easily provable.

    15. pitrou commented on Mar 21, 2011

      @pitrou
      Member

      I'm just providing this as a fast alternative to the Semaphore, which
      as far as I know, will cause a kernel call every time.

      A Semaphore might be "slow", but I'm not sure other primitives are
      faster. For the record, I tried another implementation using a critical
      section, and it's not significantly faster under a VM (even though MSDN
      claims critical sections are fast).

      Have you timed your solution?

    16. 27 remaining items

    17. kristjanvalur commented on Apr 26, 2012

      kristjanvalurmannequin
      Mannequin

      So, what do you think, should this go in? Any qualms about the thread_nt_cv.h header?

    18. pitrou commented on Apr 26, 2012

      @pitrou
      Member

      So, what do you think, should this go in? Any qualms about the thread_nt_cv.h header?

      On the principle it's ok, but I'd like to do a review before it goes
      in :)

    19. loewis commented on Apr 26, 2012

      loewismannequin
      Mannequin

      -1. Choice of operating system must be a run-time decision, not a compile-time decision. We will have to support XP for quite some time.

    20. kristjanvalur commented on Apr 27, 2012

      kristjanvalurmannequin
      Mannequin

      Antoine: of course, sorry for rushing you.

      Martin,
      This is an XP patch. The "vista" option is put in there as a compile time option, and disabled by hand. I'm not adding any apis that weren't already in use since the new gil (windows Semaphores).

      Incidentally, we should make sure that python defines NTDDI_VERSION to NTDDI_WINXP (0x05010000), either in the sources before including "windows" (tricky) or in the solution (probably in the .prefs files)

      This will ensure that we don't attempt to use non-existent features, unless we dynamically check for them.

    21. pitrou commented on Apr 27, 2012

      @pitrou
      Member

      This is an XP patch. The "vista" option is put in there as a compile
      time option, and disabled by hand. I'm not adding any apis that
      weren't already in use since the new gil (windows Semaphores).

      Martin means that you shouldn't use #ifdef's but runtime detection, so that we can provide a single installer for all Windows versions.

    22. kristjanvalur commented on Apr 27, 2012

      kristjanvalurmannequin
      Mannequin

      I understand what he meant, but that wasn't the intent of the patch. The patch is to use simulated critical sections using a semaphore, same as the new GIL implementation already does.

      If you want dynamic runtime detection, then this is a feature request :)
      I'm not sure we do it elsewhere in Python, and the benefit is doubtful...

    23. briancurtin commented on Apr 27, 2012

      @briancurtin
      Member

      We do the runtime checks for a few things in winreg as well as the os.symlink implementation and i think a few other supplemental functions for symlinking.

    24. kristjanvalur commented on Apr 27, 2012

      kristjanvalurmannequin
      Mannequin

      Ok, but the patch as provided would become more compliated. For general consumption, the primitives would need to become dynamically allocated structures, and so on. I'm not sure that its worth the effort, but I can have a look.

      (I thought the patch was radical enough, tbh.)

    25. loewis commented on Apr 30, 2012

      loewismannequin
      Mannequin

      As it stands, the patch is pointless, and can safely be rejected. We will just not have defined NTDDI_VERSION at NTDDI_VISTA for any foreseeable future, so all the Vista-specific code can be eliminated from the patch.

      Python had been using dynamic checking for API "forever". In 2.5, there was a check for presence of GetFileAttributesExA; in 2.4, there was a check for CryptAcquireContextA.

    26. kristjanvalur commented on Apr 30, 2012

      kristjanvalurmannequin
      Mannequin

      Martin, I think you misunderstand completely. the patch is _not_ about using the VISTA features. It is about not using a "mutex" for threading.lock.

      Currently, the locks in python use Mutex objects, and a WaitForSingleObjects() system call to acquire them.
      This patch replaces theses locks with user-level objects (critical sections and condition variables.). This drops the time needed for an uncontended acquire/release by 60% since there is no kernel transition and scheduling.

      The patch comes in two flavors. The current version _emulates_ condition variables on Windows by the same mechanism as I introduced for the new GIL, that is, using a combination of "critical section" objects and a construct made of a "semaphore" and a counter.

      Also provided, for those that want, and for future reference, is a version that uses native system objects (windows condition variables and SRWLocks). I can drop them from the patch to make you happy, but they are dormant and nicely show how conditional compilation can switch in more modern features for a different target architecture.

      K

      -----Original Message-----
      From: Martin v. Löwis [mailto:report@bugs.python.org]
      Sent: 30. apríl 2012 09:05
      To: Kristján Valur Jónsson
      Subject: [bpo-11618] Locks broken wrt timeouts on Windows

      Martin v. Löwis <martin@v.loewis.de> added the comment:

      As it stands, the patch is pointless, and can safely be rejected. We will just
      not have defined NTDDI_VERSION at NTDDI_VISTA for any foreseeable
      future, so all the Vista-specific code can be eliminated from the patch.

      Python had been using dynamic checking for API "forever". In 2.5, there was
      a check for presence of GetFileAttributesExA; in 2.4, there was a check for
      CryptAcquireContextA.

      ----------


      Python tracker <report@bugs.python.org>
      <http://bugs.python.org/issue11618\>


    27. kristjanvalur commented on May 2, 2012

      kristjanvalurmannequin
      Mannequin

      Again, to clarify because this seems to have been put to sleep by Martin's unfortunate dismissal. A recap of the patch:

      1. Extract the Contition Variable functions on windows out of ceval_gil.h and into thread_nt_cv.h, so that they can be used in more places.
      2. Implement the "Lock" primitive in Python using CritialSection and condition variables, rather than windows Mutexes. This gives a large performance boost on uncontended locks.
      3. Provide an alternate implementation of the Condition Variable for a build target of Vista/Server 2008, using the native contidion variable objects available for that platform.

      I think Martin got distraught by 3) and though that was the only thing this patch is about. The important part is 1) and 2) whereas 3) is provided as a bonus (and to make sure that 1) is future-safe)

      So, can we get this reviewed please?

    28. pitrou commented on May 4, 2012

      @pitrou
      Member

      I agree that Martin that it's not a good idea to add "dead" code.

      Furthermore, you patch has:

      +#ifndef _PY_EMULATED_WIN_CV
      +#define _PY_EMULATED_WIN_CV 0 /* use emulated condition variables */
      +#endif
      +
      +#if !defined NTDDI_VISTA || NTDDI_VERSION < NTDDI_VISTA
      +#undef _PY_EMULATED_WIN_CV
      +#define _PY_EMULATED_WIN_CV 1
      +#endif

      so am I right to understand that when compiled under Vista or later, it will produce an XP-incompatible binary?

    29. kristjanvalur commented on Jun 7, 2012

      kristjanvalurmannequin
      Mannequin

      Possibly the patch had a mixup
      I'm going to rework it a bit and post as a separate issue.

    30. transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      type-bugAn unexpected behavior, bug, or error

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions