You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Provide a C helper function to chain raised (but not yet caught) exceptions #67377
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee=Noneclosed_at=Nonecreated_at=<Date2015-01-08.03:17:20.994>labels= ['type-feature', '3.8']
title='Provide a C helper function to chain raised (but not yet caught) exceptions'updated_at=<Date2020-05-23.04:08:02.797>user='https://github.com/ncoghlan'
After looking into this further, PyErr_SetObject (and other APIs like PyErr_SetString which call that internally) aim to handle the chaining automatically, but they don't handle exceptions which haven't been normalized yet.
PyErr_SetObject should probably normalise the exception at the start of the call f ithe exception type is set on the thread state, but not the exception value.
changed the title [-]Convert _PyErr_ChainExceptions to a public API[/-][+]Exception chaining should trigger for non-normalised exceptions[/+]on Jan 8, 2015
Thinking about it a bit further, I suspect implicit normalisation of chained exceptions could cause problems when we only want to set __cause__ without setting __context__ (e.g. codec exception chaining).
I'm going to ponder this one for a while, but happy to hear anyone else's feedback.
At the very least, I think the C API docs could use a bit more clarity on the intricacies of C level exception chaining.
_PyErr_ChainExceptions() was added because exceptions raised in C code are not implicitly chained. The code for explicit chaining is error prone, so it was extracted in separate function. Even with _PyErr_ChainExceptions() chaining exceptions look complex. May be implicit chaining all exceptions would be better.
Chris indicates that it doesn't seem to be triggering for his patch, though (even after normalising and then restoring the exception state), and I haven't fully dug into why yet. Preliminary explorations show that the last two functional modifications were a fix for a crash bug in bpo-3611, and Antoine's original implementation of exception chaining in bpo-3108.
I've added Antoine to the nosy list, as my main takeaway at the moment is that I *don't* currently understand what's going on with the exception chaining, and I'd like to before we merge the PEP-479 patch.
The interesting discovery I made while reviewing the patch for issue
22906 is that there apparently *is* implicit chaining support in
PyErr_SetObject
Indeed, there is, and it should work properly (AFAIR there was quite a bit of debugging to make this work). Also, note that normalizing is already handled.
I'm not sure what you're witnessing that doesn't work as expected. I suspect that you may be confusing "an exception has been caught" (and is ready to be chained from) with "an exception has been raised" (i.e. PyErr_Occurred() is true).
Ah, confusion between "exception has been raised" and "we're in an active exception handler" is almost certainly what is happening. In both bpo-22906 and my previous codec exception wrapping work, we're still in the C code that raised the exception, *before* we make it back to the eval loop and any Python level exception handling.
So I think that's the distinction we need to ensure is documented, and potentially a new public helper function provided - if you're in a Python level exception handler, then exception context chaining will be handled automatically for you in PyErr_SetObject, but if you're still in C code and haven't made it back to the eval loop after raising an exception, then you're going to have to do any exception chaining explicitly.
changed the title [-]Exception chaining should trigger for non-normalised exceptions[/-][+]Provide a C helper function to chain raised (but not yet caught) exceptions[/+]on Jan 10, 2015
This helper is convenient in many cases, but it is very limited. It raises an exception with single string argument. It doesn't work in cases when the exception doesn't take arguments (PyErr_SetNone) or takes multiple or non-string arguments (PyErr_SetFromErrnoWithFilenameObject, PyErr_SetImportError). I think for public API we need more general solution. Something like this:
FTR, see PEP-490 ("Chain exceptions at C level") which proposed implicitly chaining exceptions in the PyErr_* API.
While that PEP was rejected (not all exceptions should be chained), it does make a good point about the clunkiness of using _PyErr_ChainExceptions():
PyObject*exc, *val, *tb;
PyErr_Fetch(&exc, &val, &tb);
PyErr_Format(ZipImportError, "can't open Zip file: %R", archive);
_PyErr_ChainExceptions(exc, val, tb);
So if we are going to add a public helper function, let's consider adding one that simplifies usage. For instance, how about one that indicates the next exception set should chain:
PyErr_ChainNext();
PyErr_Format(ZipImportError, "can't open Zip file: %R", archive);
Or perhaps we should revive PEP-490 with an opt out mechanism for the cases where we don't want chaining:
Many exceptions can be raised and silenced or overridden between, but we are interesting in chaining the only latest exception (or restoring the original exception if all exceptions between were silenced).
for the version we use in a few places to implicitly update the exception message, while keeping the exception type and state the same (and giving up and letting the exception through unchained if we can't work out how to do that in a reliable way).
I just want to point out one difference between _PyErr_ChainExceptions and PyErr_SetObject that I encountered while working on this issue: https://bugs.python.org/issue40696
While both functions set the context, only PyErr_SetObject does a check to prevent cycles from forming in the context chain (albeit an incomplete check, which can lead to hangs, which I mention in the issue linked above).
Activity
ncoghlan commentedon Jan 8, 2015
There's currently an internal helper API for chaining exceptions from C code, _PyErr_ChainExceptions.
It would be good to have a public API that offered equivalent functionality for use in third party C extensions.
ncoghlan commentedon Jan 8, 2015
After looking into this further, PyErr_SetObject (and other APIs like PyErr_SetString which call that internally) aim to handle the chaining automatically, but they don't handle exceptions which haven't been normalized yet.
PyErr_SetObject should probably normalise the exception at the start of the call f ithe exception type is set on the thread state, but not the exception value.
[-]Convert _PyErr_ChainExceptions to a public API[/-][+]Exception chaining should trigger for non-normalised exceptions[/+]ncoghlan commentedon Jan 8, 2015
The documentation of PyErr_SetObject, PyErr_SetString et al should also be updated to mention exception chaining.
ncoghlan commentedon Jan 8, 2015
Thinking about it a bit further, I suspect implicit normalisation of chained exceptions could cause problems when we only want to set __cause__ without setting __context__ (e.g. codec exception chaining).
I'm going to ponder this one for a while, but happy to hear anyone else's feedback.
At the very least, I think the C API docs could use a bit more clarity on the intricacies of C level exception chaining.
serhiy-storchaka commentedon Jan 8, 2015
_PyErr_ChainExceptions() was added because exceptions raised in C code are not implicitly chained. The code for explicit chaining is error prone, so it was extracted in separate function. Even with _PyErr_ChainExceptions() chaining exceptions look complex. May be implicit chaining all exceptions would be better.
ncoghlan commentedon Jan 9, 2015
The interesting discovery I made while reviewing the patch for bpo-22906 is that there apparently *is* implicit chaining support in PyErr_SetObject: https://hg.python.org/cpython/file/default/Python/errors.c#l70
Chris indicates that it doesn't seem to be triggering for his patch, though (even after normalising and then restoring the exception state), and I haven't fully dug into why yet. Preliminary explorations show that the last two functional modifications were a fix for a crash bug in bpo-3611, and Antoine's original implementation of exception chaining in bpo-3108.
I've added Antoine to the nosy list, as my main takeaway at the moment is that I *don't* currently understand what's going on with the exception chaining, and I'd like to before we merge the PEP-479 patch.
pitrou commentedon Jan 10, 2015
Indeed, there is, and it should work properly (AFAIR there was quite a bit of debugging to make this work). Also, note that normalizing is already handled.
I'm not sure what you're witnessing that doesn't work as expected. I suspect that you may be confusing "an exception has been caught" (and is ready to be chained from) with "an exception has been raised" (i.e. PyErr_Occurred() is true).
ncoghlan commentedon Jan 10, 2015
Ah, confusion between "exception has been raised" and "we're in an active exception handler" is almost certainly what is happening. In both bpo-22906 and my previous codec exception wrapping work, we're still in the C code that raised the exception, *before* we make it back to the eval loop and any Python level exception handling.
So I think that's the distinction we need to ensure is documented, and potentially a new public helper function provided - if you're in a Python level exception handler, then exception context chaining will be handled automatically for you in PyErr_SetObject, but if you're still in C code and haven't made it back to the eval loop after raising an exception, then you're going to have to do any exception chaining explicitly.
ncoghlan commentedon Jan 10, 2015
Updated the issue title and type again based on Antoine's explanation.
[-]Exception chaining should trigger for non-normalised exceptions[/-][+]Provide a C helper function to chain raised (but not yet caught) exceptions[/+]4 remaining items
serhiy-storchaka commentedon Oct 19, 2016
This helper is convenient in many cases, but it is very limited. It raises an exception with single string argument. It doesn't work in cases when the exception doesn't take arguments (PyErr_SetNone) or takes multiple or non-string arguments (PyErr_SetFromErrnoWithFilenameObject, PyErr_SetImportError). I think for public API we need more general solution. Something like this:
ericsnowcurrently commentedon May 22, 2018
FTR, see PEP-490 ("Chain exceptions at C level") which proposed implicitly chaining exceptions in the PyErr_* API.
While that PEP was rejected (not all exceptions should be chained), it does make a good point about the clunkiness of using _PyErr_ChainExceptions():
So if we are going to add a public helper function, let's consider adding one that simplifies usage. For instance, how about one that indicates the next exception set should chain:
Or perhaps we should revive PEP-490 with an opt out mechanism for the cases where we don't want chaining:
serhiy-storchaka commentedon May 22, 2018
There is usually more complex code between PyErr_Fetch() and _PyErr_ChainExceptions():
Many exceptions can be raised and silenced or overridden between, but we are interesting in chaining the only latest exception (or restoring the original exception if all exceptions between were silenced).
ericsnowcurrently commentedon May 22, 2018
good point :)
ncoghlan commentedon May 23, 2018
Also see
cpython/Objects/exceptions.c
Line 2713 in 55edd0c
cjerdonek commentedon May 22, 2020
I just want to point out one difference between _PyErr_ChainExceptions and PyErr_SetObject that I encountered while working on this issue:
https://bugs.python.org/issue40696
While both functions set the context, only PyErr_SetObject does a check to prevent cycles from forming in the context chain (albeit an incomplete check, which can lead to hangs, which I mention in the issue linked above).
cjerdonek commentedon May 23, 2020
I just posted a PR to do the above:
#20329