Skip to content

textwrap.wrap('') returns empty list #59715

Closed

Description

@cjerdonek
BPO 15510
Nosy @Yhg1s, @rhettinger, @jcea, @pitrou, @bitdancer, @cjerdonek, @ethanfurman
Files
  • issue-15510-1.patch
  • issue-15510-2.patch
  • issue-15510-3.patch: Patch keeping existing behavior.
  • issue-15510-4.patch
  • issue-15510-5.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2012-09-08.17:43:19.103>
    created_at = <Date 2012-07-31.01:26:18.819>
    labels = ['type-feature']
    title = "textwrap.wrap('') returns empty list"
    updated_at = <Date 2012-09-08.17:43:19.102>
    user = 'https://github.com/cjerdonek'

    bugs.python.org fields:

    activity = <Date 2012-09-08.17:43:19.102>
    actor = 'r.david.murray'
    assignee = 'none'
    closed = True
    closed_date = <Date 2012-09-08.17:43:19.103>
    closer = 'r.david.murray'
    components = []
    creation = <Date 2012-07-31.01:26:18.819>
    creator = 'chris.jerdonek'
    dependencies = []
    files = ['26628', '26630', '27004', '27043', '27121']
    hgrepos = []
    issue_num = 15510
    keywords = ['patch', 'needs review']
    message_count = 39.0
    messages = ['166940', '167017', '167020', '167047', '167055', '167102', '167130', '167167', '167339', '167341', '167352', '167353', '167355', '167358', '167359', '167362', '167365', '167366', '167371', '167372', '167384', '167385', '167387', '167388', '167392', '167435', '167651', '167652', '167709', '167712', '169160', '169368', '169669', '169797', '169853', '169855', '169856', '170058', '170059']
    nosy_count = 9.0
    nosy_names = ['twouters', 'gward', 'rhettinger', 'jcea', 'pitrou', 'r.david.murray', 'chris.jerdonek', 'ethan.furman', 'python-dev']
    pr_nums = []
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue15510'
    versions = ['Python 2.7', 'Python 3.2', 'Python 3.3']

    Activity

    1. cjerdonek commented on Jul 31, 2012

      @cjerdonek
      MemberAuthor

      While working on bpo-1859, I found that textwrap.wrap() returns an empty list when passed the empty string:

      >>> from textwrap import wrap
      >>> wrap('')
      []

      as opposed to a list containing the empty string which is what I expected--

      ['']

      I originally accepted this as intended behavior, but for the following reasons I now think it is a bug that we should consider fixing:

      1. The textwrap documentation says that wrap(), "Wraps the single paragraph in text (a string) so every line is at most width characters long. Returns a list of output lines, without final newlines." The empty string is less than width characters long, so following the documentation, wrapping should not change it.

      2. It is known that wrap() is broken for strings containing newlines (i.e. strings containing more than one paragraph). Indeed, this is the issue that bpo-1859 is meant to fix.

      The commonly recommended work-around is to call splitlines() on the incoming string, pass the pieces to wrap(), and then join the return values on newlines.

      However, the behavior described in this issue causes this workaround not to behave sensibly when the original string contains breaks between paragraphs. See this message, for example:

      http://bugs.python.org/issue1859#msg166627

      Currently, this work-around would return "a\nb" for both "a\nb" and "a\n\nb" for example when common sense says it should preserve the newlines and return "a\nb" and "a\n\nb", respectively.

      1. In addition, the behavior causes the following inconsistent behavior:
      >>> repr(wrap('  ', drop_whitespace=False))
      "['  ']"
      >>> repr(wrap('  ', drop_whitespace=True))
      '[]'

      The documentation says this about drop_whitespace: "If true, whitespace that, after wrapping, happens to end up at the beginning or end of a line is dropped." If the first case is correct, then, the second case should at least return a non-empty list because that is what would result from dropping whitespace from the string ' ' in the first list. This is how drop_whitespace behaves when the string contains non-whitespace characters:

      >>> repr(wrap('a  ', drop_whitespace=False))
      "['a  ']"
      >>> repr(wrap('a  ', drop_whitespace=True))
      "['a']"
      1. There is no unit test for the case of the empty string, and existing tests still pass after fixing the issue.

      If we cannot correct this behavior, then I feel we should at least document the inconsistent behavior, and then work around it in the fix for bpo-1859.

      Marking for this issue to be resolved either way before fixing bpo-1859. I am happy to prepare the patch.

    2. jcea commented on Jul 31, 2012

      @jcea
      Member

      I think this is a bug.

      Could you possibly write a patch for 2.7, 3.2 and 3.3?. With a test :-)

      I am worried about changing a behaviour that somebody is depending of. Opinions?

      Adding a couple of persons I think that could have an opinion about this.

    3. cjerdonek commented on Jul 31, 2012

      @cjerdonek
      MemberAuthor

      Could you possibly write a patch for 2.7, 3.2 and 3.3?. With a test :-)

      I would be happy to write a patch with tests (I think there may be a few edge cases we would want to test here). Should be ready within the next one or two days.

    4. cjerdonek commented on Jul 31, 2012

      @cjerdonek
      MemberAuthor

      Attaching patch with test cases.

      I added 6 new tests:

      *test_empty_string
      test_whitespace_trailing
      *test_drop_whitespace__all_whitespace
      *test_initial_indent__empty_string
      test_subsequent_indent__trailing_whitespace
      test_subsequent_indent__long_indent

      The starred ones fail with the current code (the test cases are similar to the snippets I included in my first comment). The others fail under a certain incorrect implementation of this issue. (I'm adding them to increase coverage of edge cases and to protect against future regressions.)

    5. cjerdonek commented on Jul 31, 2012

      @cjerdonek
      MemberAuthor

      Actually, here is a slightly more benign version of the patch.

      This patch makes the change in the wrap() method and leaves _wrap_chunks() alone.

      This is less intrusive because it doesn't change the behavior of _wrap_chunks(), which some people might be calling if they are subclassing TextWrapper and accessing the private API directly.

    6. cjerdonek commented on Aug 1, 2012

      @cjerdonek
      MemberAuthor

      I verified that this patch can be applied to 2.7 and that those tests pass as well.

    7. pitrou commented on Aug 1, 2012

      @pitrou
      Member

      Uh, how is this a bug? An empty text doesn't contain lines at all, so returning an empty list of lines sounds right.

      Furthermore, by "fixing" this, you may break existing software.

    8. cjerdonek commented on Aug 1, 2012

      @cjerdonek
      MemberAuthor

      Here are additional test cases impacted by this issue:

      >>> wrap("   ")
      []
      >>> wrap("\n\n\n")
      []
      >>> wrap("\n\n\n", replace_whitespace=False)
      []
      >>> wrap("  \n\n", replace_whitespace=False)
      []
    9. jcea commented on Aug 3, 2012

      @jcea
      Member

      I think this is a bug. The question to ponder is backwards compatibility, specially if this is going to be backported to 2.7/3.2.

      Chris, could you possibly ask for opinions in python-dev and/or python-list?

    10. pitrou commented on Aug 3, 2012

      @pitrou
      Member

      I think this is a bug.

      Do you have any argument?

    11. ethanfurman commented on Aug 3, 2012

      @ethanfurman
      Member

      The documentation says, "Returns a list of output lines"; an empty list is not a list of lines.

    12. ethanfurman commented on Aug 3, 2012

      @ethanfurman
      Member

      Not sure I would worry about fixing it in 2.7, although I don't have strong feelings about that.

    13. pitrou commented on Aug 3, 2012

      @pitrou
      Member

      an empty list is not a list of lines

      Really?

      >>> "".splitlines()
      []
    14. ethanfurman commented on Aug 3, 2012

      @ethanfurman
      Member
      Antoine Pitrou wrote:
      > Antoine Pitrou added the comment:
      > 
      >> an empty list is not a list of lines
      > 
      > Really?
      > 
      >>>> "".splitlines()
      > []

      Really.

      --> ''.split('\n')
      ['']

    15. 14 remaining items

    16. ethanfurman commented on Aug 8, 2012

      @ethanfurman
      Member

      Chris Jerdonek wrote:

      Here is an example on a paragraph with line breaks between paragraphs:

      s/paragraph/text/

      >>> def wrap_paras(text, width=70, **kwargs):
      ... lines = [line for para in text.splitlines()
      ... for line in wrap(para, width, **kwargs)]
      ... return "\n".join(lines)
      ...
      >>> text = """\
      ... abcd abcd
      ...
      ... abcd abcd"""
      >>> print(wrap_paras(text))
      abcd abcd
      abcd abcd

      The edge case we're discussing determines whether line breaks between paragraphs are preserved in the result. (With current behavior, they are not.)

      Having now more carefully read the docs (which, admittedly, I should
      have done before responding the first time) I found this:

      textwrap.wrap(text, width=70, **kwargs)
      	Wraps the single paragraph in text . . .
      
      textwrap.fill(text, width=70, **kwargs)
      	Wraps the single paragraph in text, . . .

      In other words, it is not designed to work on multiple paragraphs at
      once. And the documentation is fine.

      Move along, no bug to see here, move along...

    17. bitdancer commented on Aug 8, 2012

      @bitdancer
      Member

      Also you will note that the return of the empty list for an empty line is exactly what you want for wrapping multiple line-break-delimited paragraphs. Consider:

        >>> doc = "a para\nanother para\n\na third, but with an extra blank line between\n"
        >>> for line in doc.splitlines():
        ...    print('\n'.join(textwrap.wrap(line, width=5)))
        ...    if line:
        ...       print()
        a
        para

      anoth
      er
      para

      a thi
      rd,
      but
      with
      an
      extra
      blank
      line
      betwe
      en

      In other words, we need to add a blank line after our formatted paragraph, unless it is empty, in which case we don't want to add one or we'll have an extra. This assumes that single-line-paragraphs do not have blank lines between them...if they do, then the algorithm is even simpler, as you don't need the if.

    18. cjerdonek commented on Aug 26, 2012

      @cjerdonek
      MemberAuthor

      Here is a patch that documents and adds tests for the existing behavior (i.e. keeping the current behavior the same).

      I also expanded the patch slightly to cover related edge cases that involve the interplay between whitespace, empty lines, and indenting (some of which came out of the discussions above).

    19. added
      type-featureA feature request or enhancement
      and removed
      type-bugAn unexpected behavior, bug, or error
      on Aug 26, 2012
    20. cjerdonek commented on Aug 29, 2012

      @cjerdonek
      MemberAuthor

      Attaching an updated patch that improves the organization of the new test cases (test ordering, test names, and test comments, etc).

    21. bitdancer commented on Sep 1, 2012

      @bitdancer
      Member

      Added some review comments on issue-15510-4.patch.

    22. cjerdonek commented on Sep 3, 2012

      @cjerdonek
      MemberAuthor

      Thanks a lot for the very thorough review, David. I should be able to update the patch and respond to a couple of your points later today or tomorrow at the latest.

    23. cjerdonek commented on Sep 5, 2012

      @cjerdonek
      MemberAuthor

      I responded to David's comments on the review tool. Later today I will update the patch in response to his comments (accommodating all of his suggestions) along with a couple other changes.

    24. rhettinger commented on Sep 5, 2012

      @rhettinger
      Contributor

      FWIW, I agree that the existing behavior should not be changed. Most likely, a change would break some code that is currently working, and there would be little to no gain.

    25. cjerdonek commented on Sep 5, 2012

      @cjerdonek
      MemberAuthor

      Attaching an updated patch as promised in my previous comment.

      Note that I removed two edge test cases pertaining to leading whitespace. I would rather discuss those cases as part of a different issue to avoid making this thread even longer (and it is off the topic of the original issue).

    26. python-dev commented on Sep 8, 2012

      python-devmannequin
      Mannequin

      New changeset 65e5f28801e1 by R David Murray in branch '3.2':
      bpo-15510: clarify textwrap's handling of whitespace, and add confirming tests.
      http://hg.python.org/cpython/rev/65e5f28801e1

      New changeset 0e9ad455355b by R David Murray in branch 'default':
      Merge bpo-15510: clarify textwrap's handling of whitespace, and add confirming tests.
      http://hg.python.org/cpython/rev/0e9ad455355b

      New changeset fca48971ac35 by R David Murray in branch '2.7':
      bpo-15510: clarify textwrap's handling of whitespace, and add confirming tests.
      http://hg.python.org/cpython/rev/fca48971ac35

    27. bitdancer commented on Sep 8, 2012

      @bitdancer
      Member

      Thanks, Chris.

    28. transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      type-featureA feature request or enhancement

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions