You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
subprocess: check_output() doesn't close pipes on error #56703
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee=Noneclosed_at=<Date2011-09-01.21:57:18.718>created_at=<Date2011-07-04.22:13:45.149>labels= ['library']
title="subprocess: check_output() doesn't close pipes on error"updated_at=<Date2016-04-04.20:51:44.618>user='https://github.com/vstinner'
subprocess.check_output() doesn't close explicitly pipes if an error occurs. See for example issue bpo-12493 for an example of an error on .communicate().
Attached patch uses a context manager to ensure that all pipes are always closed and that the status is read to avoid zombies.
Other subprocess functions should be fixed:
call() (will fix check_call)
getstatusoutput() (will fix getoutput): see patch attached to the issue bpo-10197 to replace os.popen() by subprocess.Popen
subprocess_check_output-2.patch is a more complete patch: "fix" (?) call(), check_output() and getstatusoutput(). These functions kill the process if an exception occurs to not hang on wait() in Popen.__exit__().
Because of the kill, I don't know if the fix should be applied to 2.7 and 3.2. In case of an exception, is it better to keep the subprocess alive, or to kill it? If we keep it alive, the caller of the function cannot interact with the process, and we don't know exactly when it will finish.
By "exception", I mean unexpected exceptions: check_output() handles explicitly the TimeoutExpired exception.
But, it *probably* shouldn't be applied to 2.7& 3.2 given that it is a behaviour change.
I consider that this issue is a bug, so it should be fixed in 2.7 and
3.2. I agree that *killing* the process is a behaviour change, but we
can just close pipes on error for 2.7 and 3.2.
rosslagerwall commented on Sep 1, 2011
rosslagerwallmannequin
Mannequin
More actions
I consider that this issue is a bug, so it should be fixed in 2.7 and
3.2. I agree that *killing* the process is a behaviour change, but we
can just close pipes on error for 2.7 and 3.2.
I don't see how to test that the pipes are closed, because the Popen object (and its stdout and stderr attributes) are not visible outside the patched functions.
I consider that this issue is a bug, so it should be fixed in 2.7
and 3.2.
I tried to write a patch, but I chose to keep the code unchanged. I prefer to keep this little tricky bug, instead of introducing another more important regression.
Reopen the issue if you have a script to test the fix, or if you really want a backport.
Activity
vstinner commentedon Jul 4, 2011
subprocess.check_output() doesn't close explicitly pipes if an error occurs. See for example issue bpo-12493 for an example of an error on .communicate().
Attached patch uses a context manager to ensure that all pipes are always closed and that the status is read to avoid zombies.
Other subprocess functions should be fixed:
vstinner commentedon Jul 5, 2011
subprocess_check_output-2.patch is a more complete patch: "fix" (?) call(), check_output() and getstatusoutput(). These functions kill the process if an exception occurs to not hang on wait() in Popen.__exit__().
Because of the kill, I don't know if the fix should be applied to 2.7 and 3.2. In case of an exception, is it better to keep the subprocess alive, or to kill it? If we keep it alive, the caller of the function cannot interact with the process, and we don't know exactly when it will finish.
By "exception", I mean unexpected exceptions: check_output() handles explicitly the TimeoutExpired exception.
vstinner commentedon Jul 5, 2011
See also issue bpo-12044 which changed the context manager to call the wait() method.
rosslagerwall commentedon Sep 1, 2011
The second patch looks good. Tests?
I think it would be better to kill the process than to let it carry on.
But, it *probably* shouldn't be applied to 2.7 & 3.2 given that it is a behaviour change.
vstinner commentedon Sep 1, 2011
Ok, I will try to write a new patch with tests.
I consider that this issue is a bug, so it should be fixed in 2.7 and
3.2. I agree that *killing* the process is a behaviour change, but we
can just close pipes on error for 2.7 and 3.2.
rosslagerwall commentedon Sep 1, 2011
Yeah, that seems right.
python-dev commentedon Sep 1, 2011
New changeset 86b7f14485c9 by Victor Stinner in branch 'default':
Issue bpo-12494: Close pipes and kill process on error in subprocess functions
http://hg.python.org/cpython/rev/86b7f14485c9
vstinner commentedon Sep 1, 2011
I don't see how to test that the pipes are closed, because the Popen object (and its stdout and stderr attributes) are not visible outside the patched functions.
I tried to write a patch, but I chose to keep the code unchanged. I prefer to keep this little tricky bug, instead of introducing another more important regression.
Reopen the issue if you have a script to test the fix, or if you really want a backport.
vadmium commentedon Apr 4, 2016
As I understand it, this change only made it to 3.3+