Skip to content

test_pdb fails when only some tests are run #88126

Closed

Description

@vstinner
BPO 43960
Nosy @vstinner, @miss-islington, @iritkatriel
PRs
  • bpo-43960: reset global breakpoint state at the beginning of a pdb te… #25673
  • [3.9] bpo-43960: test_pdb resets breakpoints (GH-25673) #25681
  • [3.8] bpo-43960: test_pdb resets breakpoints (GH-25673) #25682
  • [3.9] bpo-43960: reset breakpoints to make tests deterministic (GH-25673) #25691
  • [3.8] bpo-43960: test_pdb resets breakpoints to make tests deterministic (GH-25691) #25692
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2021-04-28.15:27:55.157>
    created_at = <Date 2021-04-27.21:01:37.636>
    labels = ['3.8', 'tests', '3.9', '3.10']
    title = 'test_pdb fails when only some tests are run'
    updated_at = <Date 2021-04-28.15:43:32.273>
    user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2021-04-28.15:43:32.273>
    actor = 'vstinner'
    assignee = 'none'
    closed = True
    closed_date = <Date 2021-04-28.15:27:55.157>
    closer = 'vstinner'
    components = ['Tests']
    creation = <Date 2021-04-27.21:01:37.636>
    creator = 'vstinner'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 43960
    keywords = ['patch']
    message_count = 10.0
    messages = ['392131', '392144', '392146', '392181', '392186', '392188', '392201', '392217', '392225', '392226']
    nosy_count = 3.0
    nosy_names = ['vstinner', 'miss-islington', 'iritkatriel']
    pr_nums = ['25673', '25681', '25682', '25691', '25692']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue43960'
    versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

    Activity

    1. vstinner commented on Apr 27, 2021

      @vstinner
      MemberAuthor

      test_pdb fails with the following commands. I tested on Linux, same/similar failure on Windows.

      See also bpo-41914: "test_pdb fails".

      $ cat bisect 
      test.test_pdb.PdbTestCase.test_run_module
      test.test_pdb.test_next_until_return_at_return_event
      test.test_pdb.test_pdb_next_command_in_generator_for_loop
      
      $ ./python -m test test_pdb  --matchfile=bisect -v
      == CPython 3.10.0a7+ (heads/master:6bd9288b80, Apr 27 2021, 22:16:25) [GCC 11.0.1 20210324 (Red Hat 11.0.1-0)]
      == Linux-5.11.15-300.fc34.x86_64-x86_64-with-glibc2.33 little-endian
      == cwd: /home/vstinner/python/master/build/test_python_7720æ
      == CPU count: 8
      == encodings: locale=UTF-8, FS=utf-8
      0:00:00 load avg: 1.31 Run tests sequentially
      0:00:00 load avg: 1.31 [1/1] test_pdb
      test_run_module (test.test_pdb.PdbTestCase) ... ok
      test_next_until_return_at_return_event (test.test_pdb)
      Doctest: test.test_pdb.test_next_until_return_at_return_event ... ok
      test_pdb_next_command_in_generator_for_loop (test.test_pdb)
      Doctest: test.test_pdb.test_pdb_next_command_in_generator_for_loop ... FAIL

      ======================================================================
      FAIL: test_pdb_next_command_in_generator_for_loop (test.test_pdb)
      Doctest: test.test_pdb.test_pdb_next_command_in_generator_for_loop
      ----------------------------------------------------------------------

      Traceback (most recent call last):
        File "/home/vstinner/python/master/Lib/doctest.py", line 2205, in runTest
          raise self.failureException(self.format_failure(new.getvalue()))
      AssertionError: Failed doctest test for test.test_pdb.test_pdb_next_command_in_generator_for_loop
        File "/home/vstinner/python/master/Lib/test/test_pdb.py", line 1182, in test_pdb_next_command_in_generator_for_loop

      File "/home/vstinner/python/master/Lib/test/test_pdb.py", line 1195, in test.test_pdb.test_pdb_next_command_in_generator_for_loop
      Failed example:
      with PdbTestInput(['break test_gen',
      'continue',
      'next',
      'next',
      'next',
      'continue']):
      test_function()
      Expected:
      > <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[1]>(3)test_function()
      -> for i in test_gen():
      (Pdb) break test_gen
      Breakpoint 1 at <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[0]>:1
      (Pdb) continue
      > <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[0]>(2)test_gen()
      -> yield 0
      (Pdb) next
      value 0
      > <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[0]>(3)test_gen()
      -> return 1
      (Pdb) next
      Internal StopIteration: 1
      > <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[1]>(3)test_function()
      -> for i in test_gen():
      (Pdb) next
      > <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[1]>(5)test_function()
      -> x = 123
      (Pdb) continue
      Got:
      > <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[1]>(3)test_function()
      -> for i in test_gen():
      (Pdb) break test_gen
      Breakpoint 2 at <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[0]>:1
      (Pdb) continue
      > <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[0]>(2)test_gen()
      -> yield 0
      (Pdb) next
      value 0
      > <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[0]>(3)test_gen()
      -> return 1
      (Pdb) next
      Internal StopIteration: 1
      > <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[1]>(3)test_function()
      -> for i in test_gen():
      (Pdb) next
      > <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[1]>(5)test_function()
      -> x = 123
      (Pdb) continue

      ----------------------------------------------------------------------

      Ran 3 tests in 0.186s

      FAILED (failures=1)
      test test_pdb failed
      test_pdb failed

      == Tests result: FAILURE ==

      1 test failed:
      test_pdb

      Total duration: 480 ms
      Tests result: FAILURE

    2. iritkatriel commented on Apr 27, 2021

      @iritkatriel
      Member

      I have a fix, I'll make a PR in a bit.

      The issue here is the breakpoint number:

      Breakpoint 1 at <doctest ...

      vs

      Breakpoint 2 at <doctest ...

      The breakpoint number is global state, so tests can impact each other.

      I recently added a reset_Breakpoint() function that can be used to clear the global state at the beginning of the test: https://github.com/python/cpython/pull/21989/files

      So I will make this test use it.

    3. iritkatriel commented on Apr 27, 2021

      @iritkatriel
      Member

      I scanned the file and I think this was the only test where this was missing. test_pdb_continue_in_bottomframe doesn't have a reset, but it doesn't expect a specific breakpoint number, it does:

      Breakpoint ... at <doctest test

      So with this patch I think the tests shouldn't interfere with each other anymore.

    4. vstinner commented on Apr 28, 2021

      @vstinner
      MemberAuthor

      New changeset 21b02b5 by Irit Katriel in branch 'master':
      bpo-43960: test_pdb resets breakpoints (GH-25673)
      21b02b5

    5. vstinner commented on Apr 28, 2021

      @vstinner
      MemberAuthor

      Irit: the automated backported failed, test_pdb fails with:

      NameError: name 'reset_Breakpoint' is not defined

      Do you want to try to backport manually the change to Python 3.9 (and then I can automate the backport to 3.8).

      test_pdb on Python 3.8 and 3.9 are also affected by this issue.

    6. iritkatriel commented on Apr 28, 2021

      @iritkatriel
      Member

      We didn't backport PR21989 because it was changing the way breakpoints hare maintained, so reset_Breakpoint is not there before 3.10. I can add that to the test file manually.

    7. vstinner commented on Apr 28, 2021

      @vstinner
      MemberAuthor

      We didn't backport PR21989 because it was changing the way breakpoints hare maintained, so reset_Breakpoint is not there before 3.10. I can add that to the test file manually.

      Ah yes, it sounds reasonable to me to add reset_Breakpoint() to test_pdb with the bdb clearBreakpoints() code from master.

    8. vstinner commented on Apr 28, 2021

      @vstinner
      MemberAuthor

      New changeset 2dc6b17 by Irit Katriel in branch '3.9':
      bpo-43960: test_pdb resets breakpoints to make tests deterministic (GH-25691)
      2dc6b17

    9. vstinner commented on Apr 28, 2021

      @vstinner
      MemberAuthor

      New changeset b52cc7c by Miss Islington (bot) in branch '3.8':
      bpo-43960: test_pdb resets breakpoints to make tests deterministic (GH-25691) (GH-25692)
      b52cc7c

    10. vstinner commented on Apr 28, 2021

      @vstinner
      MemberAuthor

      Thanks for the fix Irit!

    11. transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions