PEP-654 ExceptionGroup has shipped in the 3.11 alphas, and there's an exceptiongroup backport on PyPI. We already have an issue about supporting the tracebacks (#9159) and raising ExceptionGroup from our own code (#8217).
But what happens if an ExceptionGroup is raised from test code, and it contains a special Pytest exception? Worse, what if it contains more than one kind of special exception, i.e. the test is both skipped and xfailed? e.g.
BaseExceptionGroup("msg", [ValueError(), Skipped(), XFailed()])My proposal is that XFailed should be dropped from exception groups, since it represents a non-error case. Then, if the remainder of the exception group contains a Skipped exception the test is skipped. Otherwise, we treat it as an ordinary failure (with a fancier traceback). Note that for efficiency reasons we might implement these checks in reverse order 😉
(This is a pretty niche edge case to care about, but I just sat down to handle it in Hypothesis and quickly realized that it might depend on how the test runner handles the situation.)
PEP-654
ExceptionGrouphas shipped in the 3.11 alphas, and there's anexceptiongroupbackport on PyPI. We already have an issue about supporting the tracebacks (#9159) and raisingExceptionGroupfrom our own code (#8217).But what happens if an
ExceptionGroupis raised from test code, and it contains a special Pytest exception? Worse, what if it contains more than one kind of special exception, i.e. the test is both skipped and xfailed? e.g.My proposal is that
XFailedshould be dropped from exception groups, since it represents a non-error case. Then, if the remainder of the exception group contains aSkippedexception the test is skipped. Otherwise, we treat it as an ordinary failure (with a fancier traceback). Note that for efficiency reasons we might implement these checks in reverse order 😉(This is a pretty niche edge case to care about, but I just sat down to handle it in Hypothesis and quickly realized that it might depend on how the test runner handles the situation.)