Skip to content

Restore loading of TYPE_INT64 in marshal #76192

Closed

Description

@serhiy-storchaka
BPO 32011
Nosy @serhiy-storchaka
PRs
  • bpo-32011: Revert "Issue #15480: Remove the deprecated and unused TYP… #4381
  • [3.6] bpo-32011: Revert "Issue GH-15480: Remove the deprecated and unused TYPE_INT64 code from marshal." (GH-4381) #4405
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/serhiy-storchaka'
    closed_at = <Date 2017-11-15.16:06:19.782>
    created_at = <Date 2017-11-12.17:50:16.906>
    labels = ['extension-modules', 'type-bug', '3.7']
    title = 'Restore loading of TYPE_INT64 in marshal'
    updated_at = <Date 2017-11-15.16:06:19.781>
    user = 'https://github.com/serhiy-storchaka'

    bugs.python.org fields:

    activity = <Date 2017-11-15.16:06:19.781>
    actor = 'serhiy.storchaka'
    assignee = 'serhiy.storchaka'
    closed = True
    closed_date = <Date 2017-11-15.16:06:19.782>
    closer = 'serhiy.storchaka'
    components = ['Extension Modules']
    creation = <Date 2017-11-12.17:50:16.906>
    creator = 'serhiy.storchaka'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 32011
    keywords = ['patch']
    message_count = 4.0
    messages = ['306128', '306130', '306276', '306281']
    nosy_count = 1.0
    nosy_names = ['serhiy.storchaka']
    pr_nums = ['4381', '4405']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue32011'
    versions = ['Python 3.6', 'Python 3.7']

    Activity

    1. serhiy-storchaka commented on Nov 12, 2017

      @serhiy-storchaka
      MemberAuthor

      TYPE_INT64 is supported in the marshal module since Python 1.5. Since its use causes instability of marhalled files produced on different platforms, the marshal module in Python 3.3 no longer use it when write new files (see bpo-15466). And in Python 3.4 it stopped reading files containing the TYPE_INT64 code (see bpo-15480).

      This is backward incompatible change. Python 2.7 still produce files containing the TYPE_INT64 code (though this code is rare, because integers in ranges -9223372036854775808..-2147483649 and 2147483648..9223372036854775807 are rare), and there may be marshal files produced by older versions of Python 3, Python 2 and Python 1. Formally Python 3 supports marshal formats produced by all previous versions, except the code objects and the TYPE_INT64 code.

      Supporting loading the TYPE_INT64 code don't create problems like using it for saving data. But it is needed for backward compatibility.

      I think that removing the TYPE_INT64 code was a mistake.

    2. serhiy-storchaka commented on Nov 12, 2017

      @serhiy-storchaka
      MemberAuthor

      Example:

      $ python2 -c 'import sys, marshal; marshal.dump(1234567890, sys.stdout)' | python3 -c 'import sys, marshal; print(marshal.load(sys.stdin.buffer))'
      1234567890
      $ python2 -c 'import sys, marshal; marshal.dump(123456789012345, sys.stdout)' | python3 -c 'import sys, marshal; print(marshal.load(sys.stdin.buffer))'
      Traceback (most recent call last):
        File "<string>", line 1, in <module>
      ValueError: bad marshal data (unknown type code)

      python2 -c 'import sys, marshal; marshal.dump(12345678901234567890, sys.stdout)' | python3 -c 'import sys, marshal; print(marshal.load(sys.stdin.buffer))'
      12345678901234567890

      PR 4381 reverts this change and simplifies the restored code.

    3. serhiy-storchaka commented on Nov 15, 2017

      @serhiy-storchaka
      MemberAuthor

      New changeset 00987f6 by Serhiy Storchaka in branch 'master':
      bpo-32011: Revert "Issue bpo-15480: Remove the deprecated and unused TYPE_INT64 code from marshal." (bpo-4381)
      00987f6

    4. serhiy-storchaka commented on Nov 15, 2017

      @serhiy-storchaka
      MemberAuthor

      New changeset d15bb5f by Serhiy Storchaka (Miss Islington (bot)) in branch '3.6':
      bpo-32011: Revert "Issue #59685: Remove the deprecated and unused TYPE_INT64 code from marshal." (GH-4381) (bpo-4405)
      d15bb5f

    5. transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Labels

    3.7 (EOL)end of lifeextension-modulesC modules in the Modules dirtype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions