You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Subprocess: Popen'ed children hang due to open pipes #48362
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee=Noneclosed_at=<Date2011-04-05.16:18:31.480>created_at=<Date2008-10-13.10:36:18.589>labels= ['type-bug', 'library']
title="Subprocess: Popen'ed children hang due to open pipes"updated_at=<Date2011-04-05.16:18:31.479>user='https://bugs.python.org/boye'
subprocess.Popen.wait() hangs if you have spawned multiple child
processes that do not terminate until their standard input is closed
('cat' is example of such a process). This happens on POSIX platforms.
I'm using Python 2.5.1, but I believe the issue is present in the SVN
trunk version of subprocess as well.
Here is a test program:
--------------------------
The call to p1.wait() will never return. If p2.wait is called first,
the program terminates cleanly.
p1 never terminates because p1.stdin is duplicated in the (second) child
process when the parent process forks as part of the call to p2 =
subprocess.Popen(). The line p1.stdin.close() thus has no effect.
I am not sure whether this is a bug or deliberate design, but the
behavior above seems a bit weird to me, and it took me quite some time
to figure out what happened. However, the proposed fix below of course
has side effects that may be undesirable.
wait() will not hang if the "close on exec" flag is set on the
subprocess pipes. Conveniently, the method _set_cloexec_flag already
exists to do this, so the fix amounts to calling this from
Popen._get_handles:
The suggestion looks reasonable to me, and the current behavior does not look like an intentional design.
Boye, would you be interested in proposing a patch with unit tests?
rosslagerwall commented on Jan 11, 2011
rosslagerwallmannequin
Mannequin
More actions
This issue has been fixed on 3.2.
rosslagerwall commented on Apr 5, 2011
rosslagerwallmannequin
Mannequin
More actions
This has been fixed with all the subprocess improvements in between 3.1 and 3.2 but the decision has been taken (msg125910) not to backport the fix to 3.1 and 2.7 which would involve a C extension.
However, a workaround on 2.7 and 3.1 is to set close_fds=True.
Activity
boye commentedon Oct 13, 2008
subprocess.Popen.wait() hangs if you have spawned multiple child
processes that do not terminate until their standard input is closed
('cat' is example of such a process). This happens on POSIX platforms.
I'm using Python 2.5.1, but I believe the issue is present in the SVN
trunk version of subprocess as well.
Here is a test program:
--------------------------
The call to p1.wait() will never return. If p2.wait is called first,
the program terminates cleanly.
p1 never terminates because p1.stdin is duplicated in the (second) child
process when the parent process forks as part of the call to p2 =
subprocess.Popen(). The line p1.stdin.close() thus has no effect.
I am not sure whether this is a bug or deliberate design, but the
behavior above seems a bit weird to me, and it took me quite some time
to figure out what happened. However, the proposed fix below of course
has side effects that may be undesirable.
wait() will not hang if the "close on exec" flag is set on the
subprocess pipes. Conveniently, the method _set_cloexec_flag already
exists to do this, so the fix amounts to calling this from
Popen._get_handles:
elif stdin == PIPE:
p2cread, p2cwrite = os.pipe()
self._set_cloexec_flag(p2cwrite)
The last line is the added one, similar lines must be added for stdout
and stderr.
Alternatively, the test program above will terminate cleanly if
"p1._set_cloexec_flag(p1.stdin)" is added before the creation of p2.
BreamoreBoy commentedon Jul 8, 2010
Would someone with knowledge of subprocess please comment on this.
bitdancer commentedon Jul 13, 2010
The suggestion looks reasonable to me, and the current behavior does not look like an intentional design.
Boye, would you be interested in proposing a patch with unit tests?
rosslagerwall commentedon Jan 11, 2011
This issue has been fixed on 3.2.
rosslagerwall commentedon Apr 5, 2011
This has been fixed with all the subprocess improvements in between 3.1 and 3.2 but the decision has been taken (msg125910) not to backport the fix to 3.1 and 2.7 which would involve a C extension.
However, a workaround on 2.7 and 3.1 is to set close_fds=True.
Closing as "wont fix".