You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
test_pdb fails when only some tests are run #88126
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee=Noneclosed_at=<Date2021-04-28.15:27:55.157>created_at=<Date2021-04-27.21:01:37.636>labels= ['3.8', 'tests', '3.9', '3.10']
title='test_pdb fails when only some tests are run'updated_at=<Date2021-04-28.15:43:32.273>user='https://github.com/vstinner'
Traceback (most recent call last):
File "/home/vstinner/python/master/Lib/doctest.py", line 2205, in runTestraiseself.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for test.test_pdb.test_pdb_next_command_in_generator_for_loop
File "/home/vstinner/python/master/Lib/test/test_pdb.py", line 1182, in test_pdb_next_command_in_generator_for_loop
File "/home/vstinner/python/master/Lib/test/test_pdb.py", line 1195, in test.test_pdb.test_pdb_next_command_in_generator_for_loop
Failed example:
with PdbTestInput(['break test_gen',
'continue',
'next',
'next',
'next',
'continue']):
test_function()
Expected:
> <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[1]>(3)test_function()
-> for i in test_gen():
(Pdb) break test_gen
Breakpoint 1 at <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[0]>:1
(Pdb) continue
> <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[0]>(2)test_gen()
-> yield 0
(Pdb) next
value 0
> <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[0]>(3)test_gen()
-> return 1
(Pdb) next
Internal StopIteration: 1
> <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[1]>(3)test_function()
-> for i in test_gen():
(Pdb) next
> <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[1]>(5)test_function()
-> x = 123
(Pdb) continue
Got:
> <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[1]>(3)test_function()
-> for i in test_gen():
(Pdb) break test_gen
Breakpoint 2 at <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[0]>:1
(Pdb) continue
> <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[0]>(2)test_gen()
-> yield 0
(Pdb) next
value 0
> <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[0]>(3)test_gen()
-> return 1
(Pdb) next
Internal StopIteration: 1
> <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[1]>(3)test_function()
-> for i in test_gen():
(Pdb) next
> <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[1]>(5)test_function()
-> x = 123
(Pdb) continue
I scanned the file and I think this was the only test where this was missing. test_pdb_continue_in_bottomframe doesn't have a reset, but it doesn't expect a specific breakpoint number, it does:
Breakpoint ... at <doctest test
So with this patch I think the tests shouldn't interfere with each other anymore.
We didn't backport PR21989 because it was changing the way breakpoints hare maintained, so reset_Breakpoint is not there before 3.10. I can add that to the test file manually.
We didn't backport PR21989 because it was changing the way breakpoints hare maintained, so reset_Breakpoint is not there before 3.10. I can add that to the test file manually.
Ah yes, it sounds reasonable to me to add reset_Breakpoint() to test_pdb with the bdb clearBreakpoints() code from master.
New changeset b52cc7c by Miss Islington (bot) in branch '3.8': bpo-43960: test_pdb resets breakpoints to make tests deterministic (GH-25691) (GH-25692) b52cc7c
Activity
vstinner commentedon Apr 27, 2021
test_pdb fails with the following commands. I tested on Linux, same/similar failure on Windows.
See also bpo-41914: "test_pdb fails".
$ cat bisect test.test_pdb.PdbTestCase.test_run_module test.test_pdb.test_next_until_return_at_return_event test.test_pdb.test_pdb_next_command_in_generator_for_loop $ ./python -m test test_pdb --matchfile=bisect -v == CPython 3.10.0a7+ (heads/master:6bd9288b80, Apr 27 2021, 22:16:25) [GCC 11.0.1 20210324 (Red Hat 11.0.1-0)] == Linux-5.11.15-300.fc34.x86_64-x86_64-with-glibc2.33 little-endian == cwd: /home/vstinner/python/master/build/test_python_7720æ == CPU count: 8 == encodings: locale=UTF-8, FS=utf-8 0:00:00 load avg: 1.31 Run tests sequentially 0:00:00 load avg: 1.31 [1/1] test_pdb test_run_module (test.test_pdb.PdbTestCase) ... ok test_next_until_return_at_return_event (test.test_pdb) Doctest: test.test_pdb.test_next_until_return_at_return_event ... ok test_pdb_next_command_in_generator_for_loop (test.test_pdb) Doctest: test.test_pdb.test_pdb_next_command_in_generator_for_loop ... FAIL======================================================================
FAIL: test_pdb_next_command_in_generator_for_loop (test.test_pdb)
Doctest: test.test_pdb.test_pdb_next_command_in_generator_for_loop
----------------------------------------------------------------------
File "/home/vstinner/python/master/Lib/test/test_pdb.py", line 1195, in test.test_pdb.test_pdb_next_command_in_generator_for_loop
Failed example:
with PdbTestInput(['break test_gen',
'continue',
'next',
'next',
'next',
'continue']):
test_function()
Expected:
> <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[1]>(3)test_function()
-> for i in test_gen():
(Pdb) break test_gen
Breakpoint 1 at <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[0]>:1
(Pdb) continue
> <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[0]>(2)test_gen()
-> yield 0
(Pdb) next
value 0
> <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[0]>(3)test_gen()
-> return 1
(Pdb) next
Internal StopIteration: 1
> <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[1]>(3)test_function()
-> for i in test_gen():
(Pdb) next
> <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[1]>(5)test_function()
-> x = 123
(Pdb) continue
Got:
> <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[1]>(3)test_function()
-> for i in test_gen():
(Pdb) break test_gen
Breakpoint 2 at <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[0]>:1
(Pdb) continue
> <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[0]>(2)test_gen()
-> yield 0
(Pdb) next
value 0
> <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[0]>(3)test_gen()
-> return 1
(Pdb) next
Internal StopIteration: 1
> <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[1]>(3)test_function()
-> for i in test_gen():
(Pdb) next
> <doctest test.test_pdb.test_pdb_next_command_in_generator_for_loop[1]>(5)test_function()
-> x = 123
(Pdb) continue
----------------------------------------------------------------------
Ran 3 tests in 0.186s
FAILED (failures=1)
test test_pdb failed
test_pdb failed
== Tests result: FAILURE ==
1 test failed:
test_pdb
Total duration: 480 ms
Tests result: FAILURE
iritkatriel commentedon Apr 27, 2021
I have a fix, I'll make a PR in a bit.
The issue here is the breakpoint number:
Breakpoint 1 at <doctest ...
vs
Breakpoint 2 at <doctest ...
The breakpoint number is global state, so tests can impact each other.
I recently added a reset_Breakpoint() function that can be used to clear the global state at the beginning of the test: https://github.com/python/cpython/pull/21989/files
So I will make this test use it.
iritkatriel commentedon Apr 27, 2021
I scanned the file and I think this was the only test where this was missing. test_pdb_continue_in_bottomframe doesn't have a reset, but it doesn't expect a specific breakpoint number, it does:
Breakpoint ... at <doctest test
So with this patch I think the tests shouldn't interfere with each other anymore.
vstinner commentedon Apr 28, 2021
New changeset 21b02b5 by Irit Katriel in branch 'master':
bpo-43960: test_pdb resets breakpoints (GH-25673)
21b02b5
vstinner commentedon Apr 28, 2021
Irit: the automated backported failed, test_pdb fails with:
NameError: name 'reset_Breakpoint' is not defined
Do you want to try to backport manually the change to Python 3.9 (and then I can automate the backport to 3.8).
test_pdb on Python 3.8 and 3.9 are also affected by this issue.
iritkatriel commentedon Apr 28, 2021
We didn't backport PR21989 because it was changing the way breakpoints hare maintained, so reset_Breakpoint is not there before 3.10. I can add that to the test file manually.
vstinner commentedon Apr 28, 2021
Ah yes, it sounds reasonable to me to add reset_Breakpoint() to test_pdb with the bdb clearBreakpoints() code from master.
vstinner commentedon Apr 28, 2021
New changeset 2dc6b17 by Irit Katriel in branch '3.9':
bpo-43960: test_pdb resets breakpoints to make tests deterministic (GH-25691)
2dc6b17
vstinner commentedon Apr 28, 2021
New changeset b52cc7c by Miss Islington (bot) in branch '3.8':
bpo-43960: test_pdb resets breakpoints to make tests deterministic (GH-25691) (GH-25692)
b52cc7c
vstinner commentedon Apr 28, 2021
Thanks for the fix Irit!