Skip to content

yield in list comprehensions possibly broken in 3.0 #47517

Closed

Description

@erickt
mannequin
BPO 3267
Nosy @brettcannon, @benjaminp
Superseder
  • bpo-10544: yield expression inside generator expression does nothing
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2008-07-03.17:30:59.207>
    created_at = <Date 2008-07-03.06:28:18.781>
    labels = ['interpreter-core', 'type-bug']
    title = 'yield in list comprehensions possibly broken in 3.0'
    updated_at = <Date 2010-12-08.20:12:02.175>
    user = 'https://bugs.python.org/erickt'

    bugs.python.org fields:

    activity = <Date 2010-12-08.20:12:02.175>
    actor = 'terry.reedy'
    assignee = 'none'
    closed = True
    closed_date = <Date 2008-07-03.17:30:59.207>
    closer = 'brett.cannon'
    components = ['Interpreter Core']
    creation = <Date 2008-07-03.06:28:18.781>
    creator = 'erickt'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 3267
    keywords = []
    message_count = 3.0
    messages = ['69168', '69195', '69210']
    nosy_count = 3.0
    nosy_names = ['brett.cannon', 'benjamin.peterson', 'erickt']
    pr_nums = []
    priority = 'normal'
    resolution = 'wont fix'
    stage = None
    status = 'closed'
    superseder = '10544'
    type = 'behavior'
    url = 'https://bugs.python.org/issue3267'
    versions = ['Python 3.0']

    Activity

    1. erickt commented on Jul 3, 2008

      ericktmannequin
      MannequinAuthor

      This may be a known consequence of python 3.0, but I couldn't find any
      reference to it, nor a test case that covers it. Here's a valid use of yield
      in 2.5.1:

      >>> def foo():
      ...   x=[(yield x) for x in 1,2,3]
      ...   yield 5
      ...   yield x
      >>> x=foo()
      >>> x.next()
      1
      >>> x.send(6)
      2
      >>> x.send(7)
      3
      >>> x.send(8)
      5
      >>> x.send(9)
      [6, 7, 8]
      >>> x.send(10)
      Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
      StopIteration

      But in python 3.0, this code results in:

      >>> def foo():
      ...   x=[(yield x) for x in (1,2,3)]
      ...   yield 5
      ...   yield x
      >>> x=foo()
      >>> next(x)
      5
      >>> x.send(6)
      <generator object <listcomp> at 0x3678f0>
      >>> x.send(7)
      Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
      StopIteration

      Looking further, it seems that this is a comprehension:

      >>> def foo(): [(yield 5)]
      >>> type(foo())
      <class 'generator'>

      Whereas this is not:

      >>> def foo(): [(yield 5) for x in range(3)]
      >>> type(foo())
      <class 'NoneType'>

      Is this expected behavior?

    2. benjaminp commented on Jul 3, 2008

      @benjaminp
      Contributor

      This is because list comprehensions are implemented as functions in 3.0
      so their scope is not leaked out to the rest of the function.

      Here is the bytecode for 2.6:
      >>> dis.dis(f)
        2           0 BUILD_LIST               0
                    3 DUP_TOP             
                    4 STORE_FAST               0 (_[1])
                    7 LOAD_CONST               5 ((1, 2, 3))
                   10 GET_ITER            
              >>   11 FOR_ITER                14 (to 28)
                   14 STORE_FAST               1 (x)
                   17 LOAD_FAST                0 (_[1])
                   20 LOAD_FAST                1 (x)
                   23 YIELD_VALUE         
                   24 LIST_APPEND         
                   25 JUMP_ABSOLUTE           11
              >>   28 DELETE_FAST              0 (_[1])
                   31 STORE_FAST               1 (x)

      3 34 LOAD_CONST 4 (5)
      37 YIELD_VALUE
      38 POP_TOP

      4 39 LOAD_FAST 1 (x)
      42 YIELD_VALUE
      43 POP_TOP
      44 LOAD_CONST 0 (None)
      47 RETURN_VALUE

      and here it is for 3.0:
      >>> dis.dis(f)
        2           0 LOAD_CONST               1 (<code object <listcomp> at
      0x740770, file "<stdin>", line 2>) 
                    3 MAKE_FUNCTION            0 
                    6 LOAD_CONST               6 ((1, 2, 3)) 
                    9 GET_ITER             
                   10 CALL_FUNCTION            1 
                   13 STORE_FAST               0 (x) 

      3 16 LOAD_CONST 5 (5)
      19 YIELD_VALUE
      20 POP_TOP

      4 21 LOAD_FAST 0 (x)
      24 YIELD_VALUE
      25 POP_TOP
      26 LOAD_CONST 0 (None)
      29 RETURN_VALUE

    3. brettcannon commented on Jul 3, 2008

      @brettcannon
      Member

      Yes, this change in semantics is expected. Closing as "won't fix".

    4. transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      interpreter-core(Objects, Python, Grammar, and Parser dirs)type-bugAn unexpected behavior, bug, or error

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions