You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Add a default filter for DeprecationWarning in __main__ #76156
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee='https://github.com/ncoghlan'closed_at=<Date2018-01-08.02:45:38.174>created_at=<Date2017-11-08.02:55:09.027>labels= ['type-feature', '3.7']
title='Add a default filter for DeprecationWarning in __main__'updated_at=<Date2022-03-12.06:32:30.191>user='https://github.com/ncoghlan'
This means that all deprecation warnings triggered directly by code in __main__ will start being reported again, while deprecation warnings triggered by imported modules will continue to be ignored by default.
Thus the following will start emitting DeprecationWarning by default again (as they did in 2.6 and earlier):
experiments at the REPL
directly executed scripts
modules executed with the -m switch
pkg.__main__ submodules executed with the -m switch
__main__.py files in executed directories and zip archives
However, any code *imported* from these will not trigger warnings.
This means that the following still won't trigger any warnings by default:
modules imported & functions called from entry point wrapper scripts
modules imported & functions called from pkg.__main__ submodules
modules imported & functions called from __main__.py files
The intent behind the change is that it should be much harder for developers and educators to miss seeing a deprecation warning at least once for a deprecated API, while still silencing deprecation warnings by default for deployed Python applications.
That way ad hoc scripts and the REPL will get warnings by default,
while zipapps and packages can avoid warnings by keeping their
__main__.py simple, and importing a CLI helper function from another
module. Entry point wrapper scripts will implicitly have the same
effect for installed packages.
But a lot of non-ad-hoc scripts consist of a single __main__ module, and this will produce warnings in those. Weren't those kind of scripts one of the motivators for not enabling deprecation warnings by default? I seem to remember that's where they annoyed me most, but it has been a long time (thankfully).
When we had warnings by default we got lots of complaints. Since we turned them off and turned them back on in unittest (and other test packages followed suit), we have not had complaints that I remember hearing before this thread, except about them not appearing at the REPL. Are there real-world (as opposed to theoretical) instances of the current policy causing real problems?
(And yes, I could push this as a downstream patch to Fedora's Python packages instead - if enough folks insist that springing breaking changes on people without warning them that those changes coming is fine, that's what I'll do)
Sure, it's harsh, but it's still what it looks like from the outside - there's no PEP explaining the reasoning for putting the interests of app developers that don't want to take responsibility for their user experience ahead of those of regular Python developers, and if you try to suggest that "Hey, maybe this was a mistake", you get a deluge of people yelling at you "No, we can't possibly revisit that decision".
And this is with *me* suggesting it - let alone if someone that wasn't already an experienced core dev bringing it up.
It was never about putting the interests of the app developers first, it was about putting the interests of the users first: not being pestered by seeing deprecation warnings they can't do anything about.
So yeah, maybe a PEP explaining the logic would be helpful, since there seems to be disagreement about what the logic is. I'm not in a position to write it, though.
Maybe it would indeed be worth it to push this change into Fedora/RedHat and see what the response is before doing it in core? Kind of the same principle as putting something on pypi before putting in the stdlib.
Also note that *no matter what you do* adding keywords is painful. That's why we try really really hard not to do it.
I'm currently still too annoyed to write a PEP about this - it would probably come out in tone like the state of the unicode literals PEP before I edited it.
I just don't understand how this logic is hard: *our* primary responsibility for deprecation warnings is to ensure that Python developers (including us) can alert other *Python developers* as to upcoming API breakages.
Yet somehow this primary goal has been subordinated to "app developers are bad at managing whether or not deprecation warnings are shown to their users, so it's OK that we ended up breaking deprecation warnings in general for everyone that doesn't run a test suite, even though we didn't realise that's what we were doing at the time we made the decision".
And I'll repeat: I'm entirely happy with the "revert the change for __main__ only" compromise, since it covers all the cases I care about.
If folks are also running third party single file scripts, and those scripts are emitting warnings, and they really hate seeing those warnings, then they can put "PYTHONWARNINGS=ignore::DeprecationWarning" into their shell profile.
And I'll repeat: I'm entirely happy with the "revert the change for __main__ only" compromise, since it covers all the cases I care about.
And that's what I'm unhappy with. We should either revert the change for all code, or not revert it at all.
Such a middle ground is just a wishy-washy decision-by-committee compromise. It will satisfy almost nobody (neither the people who don't want to see deprecation warnings, nor the ones who don't to miss them), is more complicated to remember, and will most certainly have to be revisited in a couple of years.
If someone else wants a different change, they can write their own
competing PEP that explains why they think mine would be worse than the
status quo.
Nobody needs a PEP to advocate for the status quo. The status quo wins
by default and you know it. It's your responsibility, if you want to
change the status quo, to convince people that the changes you are
proposing are desirable.
My impression is that you've been arguing that my changes don't go far enough (i.e. you'd prefer full reversion to the pre-2.7 behaviour), not that the status quo is superior.
My PEP covers both why I think the status quo is problematic, and why I no longer think full reversion would be a good idea (even though that was my original suggestion on python-dev). It also presents a list of additional known problems with the status quo that this particular PEP doesn't even attempt to address (although it does note some related possibilities for follow-on work if anyone is so inclined).
If you (or anyone else) would like to make the case for an alternative approach, then you either need to persuade Guido that the status quo is superior to my proposal to add a new default filter setting, *or* present a competing proposal that he prefers.
My impression is that you've been arguing that my changes don't go far enough (i.e. you'd prefer full reversion to the pre-2.7 behaviour), not that the status quo is superior.
Ideally yes. That said, I also thought that the special case you were
proposing was pointless.
Now I've changed my mind though. Your PEP makes a good convincing
argument. Thanks for writing it!
I relearned a fair bit myself in writing it - I'd completely forgotten the original rationale was about developer tools emitting unhelpful noise (rather than general Python applications), and that's far more persuasive than the rationale I'd misremembered (hence the change in my own position as the discussion went on).
I also learned we need more concrete examples of useful PYTHONWARNINGS and -W settings in the docs - I believe PEP-565 currently contains more of those than all of our existing warnings related documentation put together :)
I'll look into that "warnings cookbook" aspect as part of putting a reference patch together for the PEP.
the original rationale was about developer tools emitting unhelpful noise
(rather than general Python applications)
I don't recall that, and I'm not sure I agree (either that this is a good
reason or that that was the reason stated when we originally silenced these
by default).
Have you found a specific historic message where this is brought up as the
deciding reason? Why would noise from developer tools be worse than noise
from other applications.
My *personal* memory (as in, the reason I gave a sigh of relief when we made this change) was other tools, regardless of what core's rationale was :) (If I remember correctly it was some sysadmin-type tool I was using just about every day.)
Reviewing the original thread yet again, you're right the original decision wasn't specifically about developer tools (and I guess at some level I knew that, or I wouldn't have been motivated to suggest the change to FutureWarning's documented purpose).
That said, I consider noise from developer tools to be the most compelling example as:
Running them from the command line is their normal mode of use (unlike GUI apps which often have no visible console, implicitly silencing all warnings anyway)
It's routine for Python developers to run their developer tools on the latest version of Python, whereas regular applications are more likely to be pre-tested against a particular Python version these days
Mixing up warnings from a tool like pylint or mypy about the code it is analysing with deprecation warnings about pylint's own code is clearly going to be unhelpful and confusing, no matter how you feel about deprecation warnings in the general case
All the points that are most debatable for regular apps ("Do they even have a console?", "Why are end users supplying their own Python version?", "Why can't they just ignore the error?") have clear answers in the developer tools case that favour leaving deprecation warnings off by default.
I simply don't think developer tools are the prime example of command line
tools (it's just that because we are Python developers here, *everything*
we use tends to be a developer tool). And "app" does not mean or imply "GUI
app".
The default command line UX of Python developer tools is also more unambiguously python-dev's responsibility, though. That's where my own initial line of argument for full reversion (i.e. that we should be letting app developers manage their own stderr UX, not trying to manage it for them) fell down.
I think #4458 is ready to go now, but I expect it would benefit from a review before I merge it. If anyone has the time to take a look, it would be much appreciated :)
However, I'd also like to get it in for 3.7.0a4 this week, so I'll merge it tomorrow regardless - if we decide we need to tweak it before the first beta, we can do that.
Activity
ncoghlan commentedon Nov 8, 2017
As per the post at https://mail.python.org/pipermail/python-dev/2017-November/150366.html, this is an RFE to cover adding a new warning filter to the default set:
This means that all deprecation warnings triggered directly by code in __main__ will start being reported again, while deprecation warnings triggered by imported modules will continue to be ignored by default.
Thus the following will start emitting DeprecationWarning by default again (as they did in 2.6 and earlier):
However, any code *imported* from these will not trigger warnings.
This means that the following still won't trigger any warnings by default:
The intent behind the change is that it should be much harder for developers and educators to miss seeing a deprecation warning at least once for a deprecated API, while still silencing deprecation warnings by default for deployed Python applications.
merwok commentedon Nov 8, 2017
This can’t be backported, but could the docs of 2.7 and stable 3.x version gain an example of equivalent PYTHONWARNINGS envvar?
pitrou commentedon Nov 8, 2017
This is backwards. If there's one place where you *don't* want stderr polluted with warnings (or any kind of logging), it's an interactive prompt.
More generally, I think this entire proposal is unsound, and needs a PEP written to make its case clearly.
bitdancer commentedon Nov 8, 2017
From the linked email:
But a lot of non-ad-hoc scripts consist of a single __main__ module, and this will produce warnings in those. Weren't those kind of scripts one of the motivators for not enabling deprecation warnings by default? I seem to remember that's where they annoyed me most, but it has been a long time (thankfully).
When we had warnings by default we got lots of complaints. Since we turned them off and turned them back on in unittest (and other test packages followed suit), we have not had complaints that I remember hearing before this thread, except about them not appearing at the REPL. Are there real-world (as opposed to theoretical) instances of the current policy causing real problems?
ncoghlan commentedon Nov 9, 2017
Yes - stuff just flat out breaks when Linux distros upgrade Python.
The response is "Yeah, python-dev decided years ago that they don't care about that".
ncoghlan commentedon Nov 9, 2017
(And yes, I could push this as a downstream patch to Fedora's Python packages instead - if enough folks insist that springing breaking changes on people without warning them that those changes coming is fine, that's what I'll do)
gvanrossum commentedon Nov 9, 2017
That's a little harsh don't you think?
ncoghlan commentedon Nov 9, 2017
Sure, it's harsh, but it's still what it looks like from the outside - there's no PEP explaining the reasoning for putting the interests of app developers that don't want to take responsibility for their user experience ahead of those of regular Python developers, and if you try to suggest that "Hey, maybe this was a mistake", you get a deluge of people yelling at you "No, we can't possibly revisit that decision".
And this is with *me* suggesting it - let alone if someone that wasn't already an experienced core dev bringing it up.
bitdancer commentedon Nov 9, 2017
It was never about putting the interests of the app developers first, it was about putting the interests of the users first: not being pestered by seeing deprecation warnings they can't do anything about.
So yeah, maybe a PEP explaining the logic would be helpful, since there seems to be disagreement about what the logic is. I'm not in a position to write it, though.
Maybe it would indeed be worth it to push this change into Fedora/RedHat and see what the response is before doing it in core? Kind of the same principle as putting something on pypi before putting in the stdlib.
Also note that *no matter what you do* adding keywords is painful. That's why we try really really hard not to do it.
ncoghlan commentedon Nov 9, 2017
I'm currently still too annoyed to write a PEP about this - it would probably come out in tone like the state of the unicode literals PEP before I edited it.
I just don't understand how this logic is hard: *our* primary responsibility for deprecation warnings is to ensure that Python developers (including us) can alert other *Python developers* as to upcoming API breakages.
Yet somehow this primary goal has been subordinated to "app developers are bad at managing whether or not deprecation warnings are shown to their users, so it's OK that we ended up breaking deprecation warnings in general for everyone that doesn't run a test suite, even though we didn't realise that's what we were doing at the time we made the decision".
And I'll repeat: I'm entirely happy with the "revert the change for __main__ only" compromise, since it covers all the cases I care about.
If folks are also running third party single file scripts, and those scripts are emitting warnings, and they really hate seeing those warnings, then they can put "PYTHONWARNINGS=ignore::DeprecationWarning" into their shell profile.
pitrou commentedon Nov 9, 2017
And that's what I'm unhappy with. We should either revert the change for all code, or not revert it at all.
Such a middle ground is just a wishy-washy decision-by-committee compromise. It will satisfy almost nobody (neither the people who don't want to see deprecation warnings, nor the ones who don't to miss them), is more complicated to remember, and will most certainly have to be revisited in a couple of years.
gvanrossum commentedon Nov 9, 2017
Fine, but since *I* won't tolerate these warnings on for everything, this is the best you can do. If you don't like it, the status quo wins.
pitrou commentedon Nov 9, 2017
Then let the statu quo wins!
9 remaining items
pitrou commentedon Nov 12, 2017
Le 12/11/2017 à 02:17, Nick Coghlan a écrit :
Nobody needs a PEP to advocate for the status quo. The status quo wins
by default and you know it. It's your responsibility, if you want to
change the status quo, to convince people that the changes you are
proposing are desirable.
ncoghlan commentedon Nov 12, 2017
My impression is that you've been arguing that my changes don't go far enough (i.e. you'd prefer full reversion to the pre-2.7 behaviour), not that the status quo is superior.
My PEP covers both why I think the status quo is problematic, and why I no longer think full reversion would be a good idea (even though that was my original suggestion on python-dev). It also presents a list of additional known problems with the status quo that this particular PEP doesn't even attempt to address (although it does note some related possibilities for follow-on work if anyone is so inclined).
If you (or anyone else) would like to make the case for an alternative approach, then you either need to persuade Guido that the status quo is superior to my proposal to add a new default filter setting, *or* present a competing proposal that he prefers.
pitrou commentedon Nov 12, 2017
Le 12/11/2017 à 12:40, Nick Coghlan a écrit :
Ideally yes. That said, I also thought that the special case you were
proposing was pointless.
Now I've changed my mind though. Your PEP makes a good convincing
argument. Thanks for writing it!
ncoghlan commentedon Nov 12, 2017
I relearned a fair bit myself in writing it - I'd completely forgotten the original rationale was about developer tools emitting unhelpful noise (rather than general Python applications), and that's far more persuasive than the rationale I'd misremembered (hence the change in my own position as the discussion went on).
ncoghlan commentedon Nov 12, 2017
I also learned we need more concrete examples of useful PYTHONWARNINGS and -W settings in the docs - I believe PEP-565 currently contains more of those than all of our existing warnings related documentation put together :)
I'll look into that "warnings cookbook" aspect as part of putting a reference patch together for the PEP.
gvanrossum commentedon Nov 12, 2017
I don't recall that, and I'm not sure I agree (either that this is a good
reason or that that was the reason stated when we originally silenced these
by default).
Have you found a specific historic message where this is brought up as the
deciding reason? Why would noise from developer tools be worse than noise
from other applications.
bitdancer commentedon Nov 12, 2017
My *personal* memory (as in, the reason I gave a sigh of relief when we made this change) was other tools, regardless of what core's rationale was :) (If I remember correctly it was some sysadmin-type tool I was using just about every day.)
ncoghlan commentedon Nov 12, 2017
Reviewing the original thread yet again, you're right the original decision wasn't specifically about developer tools (and I guess at some level I knew that, or I wouldn't have been motivated to suggest the change to FutureWarning's documented purpose).
That said, I consider noise from developer tools to be the most compelling example as:
All the points that are most debatable for regular apps ("Do they even have a console?", "Why are end users supplying their own Python version?", "Why can't they just ignore the error?") have clear answers in the developer tools case that favour leaving deprecation warnings off by default.
gvanrossum commentedon Nov 12, 2017
I simply don't think developer tools are the prime example of command line
tools (it's just that because we are Python developers here, *everything*
we use tends to be a developer tool). And "app" does not mean or imply "GUI
app".
ncoghlan commentedon Nov 13, 2017
The default command line UX of Python developer tools is also more unambiguously python-dev's responsibility, though. That's where my own initial line of argument for full reversion (i.e. that we should be letting app developers manage their own stderr UX, not trying to manage it for them) fell down.
ncoghlan commentedon Jan 7, 2018
I think #4458 is ready to go now, but I expect it would benefit from a review before I merge it. If anyone has the time to take a look, it would be much appreciated :)
However, I'd also like to get it in for 3.7.0a4 this week, so I'll merge it tomorrow regardless - if we decide we need to tweak it before the first beta, we can do that.
ncoghlan commentedon Jan 8, 2018
New changeset 9b99747 by Nick Coghlan in branch 'master':
bpo-31975 (PEP-565): Show DeprecationWarning in __main__ (GH-4458)
9b99747