Skip to content

test_asyncio leaks dangling threads #75433

Closed

Description

@vstinner
BPO 31250
Nosy @vstinner, @1st1
PRs
  • bpo-31250, test_asyncio: fix dangling threads #3252
  • bpo-31250, test_asyncio: fix EventLoopTestsMixin.tearDown() #3264
  • [3.6] bpo-31250: test_asyncio: fix dangling threads #3517
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2017-09-12.21:18:39.212>
    created_at = <Date 2017-08-21.16:33:56.001>
    labels = ['3.7', 'expert-asyncio', 'tests', 'performance']
    title = 'test_asyncio leaks dangling threads'
    updated_at = <Date 2017-09-13.09:56:51.402>
    user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2017-09-13.09:56:51.402>
    actor = 'vstinner'
    assignee = 'none'
    closed = True
    closed_date = <Date 2017-09-12.21:18:39.212>
    closer = 'vstinner'
    components = ['Tests', 'asyncio']
    creation = <Date 2017-08-21.16:33:56.001>
    creator = 'vstinner'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 31250
    keywords = ['patch']
    message_count = 7.0
    messages = ['300637', '301048', '301097', '301099', '301131', '301155', '301995']
    nosy_count = 2.0
    nosy_names = ['vstinner', 'yselivanov']
    pr_nums = ['3252', '3264', '3517']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'resource usage'
    url = 'https://bugs.python.org/issue31250'
    versions = ['Python 3.6', 'Python 3.7']

    Activity

    1. vstinner commented on Aug 21, 2017

      @vstinner
      MemberAuthor

      Using PR 3138 of bpo-31234, I noticed that test_asyncio leaks dangling threads.

      Tests which randomly leak threads:
      ---
      test.test_asyncio.test_base_events.BaseEventLoopWithSelectorTests.test_create_connection_no_inet_pton
      test.test_asyncio.test_base_events.BaseEventLoopWithSelectorTests.test_create_connection_service_name
      test.test_asyncio.test_base_events.BaseEventLoopWithSelectorTests.test_create_connection_bluetooth
      test.test_asyncio.test_base_events.BaseEventLoopWithSelectorTests.test_create_datagram_endpoint_no_addrinfo
      test.test_asyncio.test_futures.DuckTests.test_wrap_future
      test.test_asyncio.test_futures.CFutureTests.test_wrap_future
      test.test_asyncio.test_futures.CFutureTests.test_wrap_future_use_global_loop
      test.test_asyncio.test_futures.PyFutureTests.test_wrap_future
      test.test_asyncio.test_futures.PyFutureTests.test_wrap_future_use_global_loop
      test.test_asyncio.test_selector_events.BaseSelectorEventLoopTests.test_sock_connect_resolve_using_socket_params
      test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests.test_run_coroutine_threadsafe
      test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests.test_run_coroutine_threadsafe_with_timeout
      test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests.test_run_coroutine_threadsafe_task_cancelled
      test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests.test_run_coroutine_threadsafe_task_factory_exception
      ---

      Write this list into "tests" and run "python3 -m test --matchfile=tests".

      haypo@selma$ ./python -u -m test -v test_asyncio --matchfile=tests
      == CPython 3.7.0a0 (heads/concurrent_futures:f277fa3, Aug 21 2017, 18:11:41) [GCC 6.4.1 20170727 (Red Hat 6.4.1-1)]
      (...)
      test_create_connection_bluetooth (test.test_asyncio.test_base_events.BaseEventLoopWithSelectorTests) ... ok
      test_create_connection_no_inet_pton (test.test_asyncio.test_base_events.BaseEventLoopWithSelectorTests) ... Warning -- threading_cleanup() detected 4 leaked threads (count: 4, dangling: 5)
      ok
      test_create_connection_service_name (test.test_asyncio.test_base_events.BaseEventLoopWithSelectorTests) ... Warning -- threading_cleanup() detected 4 leaked threads (count: 4, dangling: 5)
      ok
      test_create_datagram_endpoint_no_addrinfo (test.test_asyncio.test_base_events.BaseEventLoopWithSelectorTests) ... Warning -- threading_cleanup() detected 1 leaked threads (count: 1, dangling: 2)
      ok
      test_wrap_future (test.test_asyncio.test_futures.CFutureTests) ... Warning -- threading_cleanup() detected 1 leaked threads (count: 1, dangling: 2)
      ok
      test_wrap_future_use_global_loop (test.test_asyncio.test_futures.CFutureTests) ... Warning -- threading_cleanup() detected 1 leaked threads (count: 1, dangling: 2)
      ok
      test_wrap_future (test.test_asyncio.test_futures.DuckTests) ... ok
      test_wrap_future (test.test_asyncio.test_futures.PyFutureTests) ... Warning -- threading_cleanup() detected 1 leaked threads (count: 1, dangling: 2)
      ok
      test_wrap_future_use_global_loop (test.test_asyncio.test_futures.PyFutureTests) ... Warning -- threading_cleanup() detected 1 leaked threads (count: 1, dangling: 2)
      ok
      test_sock_connect_resolve_using_socket_params (test.test_asyncio.test_selector_events.BaseSelectorEventLoopTests) ... Warning -- threading_cleanup() detected 1 leaked threads (count: 1, dangling: 2)
      ok
      test_run_coroutine_threadsafe (test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests)
      Test coroutine submission from a thread to an event loop. ... ok
      test_run_coroutine_threadsafe_task_cancelled (test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests)
      Test coroutine submission from a tread to an event loop ... ok
      test_run_coroutine_threadsafe_task_factory_exception (test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests)
      Test coroutine submission from a tread to an event loop ... Warning -- threading_cleanup() detected 1 leaked threads (count: 1, dangling: 2)
      ok
      test_run_coroutine_threadsafe_with_timeout (test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests)
      Test coroutine submission from a thread to an event loop ... ok
      (...)

    2. vstinner commented on Aug 31, 2017

      @vstinner
      MemberAuthor

      I proposed PR 3252 to only fix unit tests, but not asyncio.

      An alternative is to modify BaseEventLoop.close() to replace executor.shutdown(wait=False) with executor.shutdown(wait=True), but it would change the behaviour. I'm not sure if we want to wait until all threads complete on close()?

    3. vstinner commented on Sep 1, 2017

      @vstinner
      MemberAuthor

      New changeset 16432be by Victor Stinner in branch 'master':
      bpo-31250, test_asyncio: fix dangling threads (bpo-3252)
      16432be

    4. vstinner commented on Sep 1, 2017

      @vstinner
      MemberAuthor

      I picked the safe option: only modify unit tests.

    5. vstinner commented on Sep 1, 2017

      @vstinner
      MemberAuthor

      Hum, EventLoopTestsMixin.tearDown() still has a bug. I reopen the issue.

    6. vstinner commented on Sep 1, 2017

      @vstinner
      MemberAuthor

      New changeset e8a533f by Victor Stinner in branch 'master':
      bpo-31250, test_asyncio: fix EventLoopTestsMixin.tearDown() (bpo-3264)
      e8a533f

    7. vstinner commented on Sep 12, 2017

      @vstinner
      MemberAuthor

      New changeset 5013a5e by Victor Stinner in branch '3.6':
      [3.6] bpo-31250: test_asyncio: fix dangling threads (bpo-3517)
      5013a5e

    8. transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions