Skip to content

cPickle corner case - docs or bug? #46548

Closed

Description

@gpk
mannequin
BPO 2295
Nosy @birkenfeld, @amauryfa, @llucax, @avassalotti

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = 'https://github.com/avassalotti'
closed_at = <Date 2013-12-01.21:44:07.163>
created_at = <Date 2008-03-15.19:18:44.302>
labels = ['type-bug', 'library']
title = 'cPickle corner case - docs or bug?'
updated_at = <Date 2013-12-01.21:44:07.160>
user = 'https://bugs.python.org/gpk'

bugs.python.org fields:

activity = <Date 2013-12-01.21:44:07.160>
actor = 'alexandre.vassalotti'
assignee = 'alexandre.vassalotti'
closed = True
closed_date = <Date 2013-12-01.21:44:07.163>
closer = 'alexandre.vassalotti'
components = ['Library (Lib)']
creation = <Date 2008-03-15.19:18:44.302>
creator = 'gpk'
dependencies = []
files = []
hgrepos = []
issue_num = 2295
keywords = []
message_count = 5.0
messages = ['63560', '63629', '68289', '68292', '204970']
nosy_count = 5.0
nosy_names = ['gpk', 'georg.brandl', 'amaury.forgeotdarc', 'llucax', 'alexandre.vassalotti']
pr_nums = []
priority = 'normal'
resolution = 'works for me'
stage = 'test needed'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue2295'
versions = ['Python 2.7']

Activity

  1. gpk commented on Mar 15, 2008

    gpkmannequin
    MannequinAuthor

    If you attempt to cPickle a class, cPickle checks
    that it can get the identical class by importing it.

    If that check fails, it reports:
    Traceback (most recent call last):
    ...
    "/usr/local/lib/python2.5/site-packages/newstem2-0.12.3-py2.5-linux-i686.egg/newstem2/stepperclient.py",
    line 41, in send
        s = cPickle.dumps( args, cPickle.HIGHEST_PROTOCOL)
    cPickle.PicklingError: Can't pickle <class 'test_simple2.aModel'>: it's
    not the same object as test_simple2.aModel
    $ 

    Normally, this is probably a good thing. However, if you do
    an import using the "imp" module, via
    imp.load_module(name, fd, pn, desc), you get the "same" module
    containing the "same" classes, but everything is duplicated at
    different addresses. In other words, you get distinct class
    objects from what cPickle
    will find. Consequently, the when cPickle makes the
    "is" comparison between what you gave it and what it can find,
    it will fail and cause an error.

    In this case, the error is wrong. I know that the aModel classes
    come from the same file and are member-for-member the same.

    This may well be a documentation
    error: it needs to mention this test and note that classes in modules
    imported
    via imp are not picklable. Or, imp needs to note that its results
    are not picklable. Or both.

    Or, maybe it's something that should be fixed, though I'm not
    sure if there is a general solution that will always behave
    well.

  2. added
    docsDocumentation in the Doc dir
    stdlibStandard Library Python modules in the Lib/ directory
    type-bugAn unexpected behavior, bug, or error
    on Mar 15, 2008
  3. amauryfa commented on Mar 17, 2008

    @amauryfa
    Contributor

    The following "Works for me":

    >>> import imp, cPickle
    >>> mymod = imp.load_module('mymod', *imp.find_module('codecs'))
    >>> cPickle.dumps(mymod.Codec(), cPickle.HIGHEST_PROTOCOL)
    '\x80\x02(cmymod\nCodec\nq\x01o}q\x02b.'

    Do you have a short test case to reproduce your problem?
    Does your code tweak sys.modules?

  4. llucax commented on Jun 16, 2008

    llucaxmannequin
    Mannequin

    I'm having the same problem here.

    Error is: cPickle.PicklingError: Can't pickle qos.Device: it's not the
    same object as qos.Device

    If I use pickle module, it works fine. If I use cPickle module but with
    protocol=0, it works fine (protocol=1 or protocol=2 both fail).

    I'm trying to make a simple example that reproduce the problem, but I
    couldn't do it yet (small tests seems to work). So for now, I just can
    only offer the complete source code of my project[1] (see branch
    "python-bug-2295"). Not that the bug is exposed by the commit
    b0ef5dd[2], and it's only exposed in the "qos" module (other modules
    works fine).

    To see the backtrace it should be enough to download the project (you
    can download a tarball[3] or use git via git protocol[4] -preferred- or
    http[5]) and execute ./pymind

    [1]
    http://git.llucax.com.ar/?p=software/pymin.git;a=shortlog;h=refs/heads/python-bug-2295
    [2]
    http://git.llucax.com.ar/?p=software/pymin.git;a=commitdiff;h=b0ef5ddfd5b04ecb09a18fb6f253c9f8a7db48a8
    [3]
    http://git.llucax.com.ar/?p=software/pymin.git;a=snapshot;h=f19fc5b4f5eeda18a24b1f5a2d042fd206c9ed0f;sf=tgz
    [4] git clone git://git.llucax.com.ar/software/pymin.git
    [5] git clone http://git.llucax.com.ar/git/software/pymin.git

  5. llucax commented on Jun 16, 2008

    llucaxmannequin
    Mannequin

    I've noted that the problem goes away if I move the pymin/services/*
    directories (i.e., the packages in pymin/services) to another place (for
    example, a services directory in the root of the project).

    I still can't make a simple test to reproduce the problem.

  6. avassalotti commented on Dec 1, 2013

    @avassalotti
    Member

    We can't fix this without a working test case. Feel free to re-open if you find one.

  7. transferred this issue fromon Apr 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

stdlibStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or error

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions