Skip to content

Cannot build importlib.h on Windows #59636

Closed

Description

@amauryfa
BPO 15431
Nosy @loewis, @amauryfa, @pitrou, @kristjanvalur, @jkloth, @briancurtin
Files
  • _freeze_importlib.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2012-07-28.20:05:54.676>
    created_at = <Date 2012-07-23.06:21:49.930>
    labels = ['build', 'OS-windows', 'release-blocker']
    title = 'Cannot build importlib.h on Windows'
    updated_at = <Date 2012-07-28.20:05:54.675>
    user = 'https://github.com/amauryfa'

    bugs.python.org fields:

    activity = <Date 2012-07-28.20:05:54.675>
    actor = 'loewis'
    assignee = 'none'
    closed = True
    closed_date = <Date 2012-07-28.20:05:54.676>
    closer = 'loewis'
    components = ['Build', 'Windows']
    creation = <Date 2012-07-23.06:21:49.930>
    creator = 'amaury.forgeotdarc'
    dependencies = []
    files = ['26538']
    hgrepos = []
    issue_num = 15431
    keywords = ['patch']
    message_count = 14.0
    messages = ['166209', '166425', '166554', '166560', '166582', '166587', '166589', '166674', '166675', '166676', '166684', '166685', '166687', '166688']
    nosy_count = 8.0
    nosy_names = ['loewis', 'amaury.forgeotdarc', 'pitrou', 'kristjan.jonsson', 'jkloth', 'brian.curtin', 'python-dev', 'sbt']
    pr_nums = []
    priority = 'release blocker'
    resolution = 'fixed'
    stage = None
    status = 'closed'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue15431'
    versions = ['Python 3.3']

    Activity

    1. amauryfa commented on Jul 23, 2012

      @amauryfa
      ContributorAuthor

      On Windows, the _freeze_importlib tool is not built, so it's not possible to refresh the file Python/importlib.h, which makes development on the importlib very difficult on Windows.

      The Makefile contains the rules below, it's probably necessary to implement a similar behavior in a new build project:

      Modules/_freeze_importlib: Modules/_freeze_importlib.o $(LIBRARY_OBJS_OMIT_FROZEN)
      $(LINKCC) $(PY_LDFLAGS) -o $@ Modules/_freeze_importlib.o $(LIBRARY_OBJS_OMIT_FROZEN) $(LIBS) $(MODLIBS) $(SYSLIBS) $(LDLAST)

      Python/importlib.h: $(srcdir)/Lib/importlib/_bootstrap.py Modules/_freeze_importlib.c
      $(MAKE) Modules/_freeze_importlib
      ./Modules/_freeze_importlib \
      $(srcdir)/Lib/importlib/_bootstrap.py Python/importlib.h

    2. kristjanvalur commented on Jul 25, 2012

      kristjanvalurmannequin
      Mannequin

      shouldn't be too hard to add, I'll give it a go soon.

    3. kristjanvalur commented on Jul 27, 2012

      kristjanvalurmannequin
      Mannequin

      Here's a patch. A new project, _freeze_importlib, has the necessary prerequisites and runs the executable, generating importlib.h, as a post-build step.

      I'm not sure if we should make this project a pre-requisite of any other projects, though

    4. amauryfa commented on Jul 27, 2012

      @amauryfa
      ContributorAuthor

      IMO dependencies is not a big issue here. In the worst case, the developer can build a second time...

      I think your patch won't work on Unix machines: _freeze_importlib.c is linked with all Python files *except* frozen.c which defines PyImport_FrozenModules. #ifdef should be used IMO.

      Also, I noticed that the marshalled code is different. The first difference a 32bit vs. 64bit issue: 0xFFFFFFFF is marshalled as a TYPE_INT64 on 64bit platforms, but as a TYPE_LONG on 32bit platforms. See bpo-15466.

    5. kristjanvalur commented on Jul 27, 2012

      kristjanvalurmannequin
      Mannequin

      Thank you. I see how I must change the linkage on PC Only, although I wonder why tuen unix version isn't simply doing something similar..

      The marshaled code was indeed created using 64 bits. I was unsure whether to include that file as part of the patch, perhaps I shouldn't.

      I'll wait for an update for the 64 bit issue.

    6. amauryfa commented on Jul 27, 2012

      @amauryfa
      ContributorAuthor

      (Sorry for the previous message, it was wrong, and I also pressed the wrong button...)

      A Unix Makefile cannot stand circular dependencies, so _freezeimportlib.c cannot depend on frozen.o. OTOH it's not simple to have almost-similar list of files on VS projects (here we need all-of-python-but-frozen.c), so your solution is not bad either.
      Please check that python33.dll is not always rebuilt each time you hit F7, though.

    7. pitrou commented on Jul 27, 2012

      @pitrou
      Member

      I'll wait for an update for the 64 bit issue.

      I don't think it's necessary, it's a pretty minor issue since the generated bytecode is portable anyway.

    8. python-dev commented on Jul 28, 2012

      python-devmannequin
      Mannequin

      New changeset fe29a657bde9 by Martin v. Löwis in branch 'default':
      Issue bpo-15431: Add _freeze_importlib project to regenerate importlib.h on Windows.
      http://hg.python.org/cpython/rev/fe29a657bde9

    9. loewis commented on Jul 28, 2012

      loewismannequin
      Mannequin

      Thanks for the patch, committed with slight modifications.

    10. amauryfa commented on Jul 28, 2012

      @amauryfa
      ContributorAuthor

      Comments about PyImport_FrozenModules linkage have not been addressed. Now the build is failing on all Unix machines...

    11. pitrou commented on Jul 28, 2012

      @pitrou
      Member

      To be clear, the build fails when importlib needs to be frozen again:

      $ touch Lib/importlib/_bootstrap.py 
      $ make
      make Modules/_freeze_importlib
      make[1] : on entre dans le répertoire « /home/antoine/cpython/default »
      gcc -pthread -c -Wno-unused-result -g -O0 -Wall -Wstrict-prototypes    -I. -I./Include    -DPy_BUILD_CORE -o Modules/_freeze_importlib.o Modules/_freeze_importlib.c
      gcc -pthread   -o Modules/_freeze_importlib Modules/_freeze_importlib.o Modules/getbuildinfo.o Parser/acceler.o Parser/grammar1.o Parser/listnode.o Parser/node.o Parser/parser.o Parser/bitset.o Parser/metagrammar.o Parser/firstsets.o Parser/grammar.o Parser/pgen.o Parser/myreadline.o Parser/parsetok.o Parser/tokenizer.o Objects/abstract.o Objects/accu.o Objects/boolobject.o Objects/bytes_methods.o Objects/bytearrayobject.o Objects/bytesobject.o Objects/cellobject.o Objects/classobject.o Objects/codeobject.o Objects/complexobject.o Objects/descrobject.o Objects/enumobject.o Objects/exceptions.o Objects/genobject.o Objects/fileobject.o Objects/floatobject.o Objects/frameobject.o Objects/funcobject.o Objects/iterobject.o Objects/listobject.o Objects/longobject.o Objects/dictobject.o Objects/memoryobject.o Objects/methodobject.o Objects/moduleobject.o Objects/namespaceobject.o Objects/object.o Objects/obmalloc.o Objects/capsule.o Objects/rangeobject.o Objects/setobject.o Objects/sliceobject.o Objects/structseq.o Objects/tupleobject.o Objects/typeobject.o Objects/unicodeobject.o Objects/unicodectype.o Objects/weakrefobject.o Python/_warnings.o Python/Python-ast.o Python/asdl.o Python/ast.o Python/bltinmodule.o Python/ceval.o Python/compile.o Python/codecs.o Python/dynamic_annotations.o Python/errors.o Python/frozenmain.o Python/future.o Python/getargs.o Python/getcompiler.o Python/getcopyright.o Python/getplatform.o Python/getversion.o Python/graminit.o Python/import.o Python/importdl.o Python/marshal.o Python/modsupport.o Python/mystrtoul.o Python/mysnprintf.o Python/peephole.o Python/pyarena.o Python/pyctype.o Python/pyfpe.o Python/pymath.o Python/pystate.o Python/pythonrun.o Python/pytime.o Python/random.o Python/structmember.o Python/symtable.o Python/sysmodule.o Python/traceback.o Python/getopt.o Python/pystrcmp.o Python/pystrtod.o Python/dtoa.o Python/formatter_unicode.o Python/fileutils.o Python/dynload_shlib.o   Python/thread.o Modules/config.o Modules/getpath.o Modules/main.o Modules/gcmodule.o  Modules/_threadmodule.o  Modules/signalmodule.o  Modules/posixmodule.o  Modules/errnomodule.o  Modules/pwdmodule.o  Modules/_sre.o  Modules/_codecsmodule.o  Modules/_weakref.o  Modules/_functoolsmodule.o  Modules/operator.o  Modules/_collectionsmodule.o  Modules/itertoolsmodule.o  Modules/_localemodule.o  Modules/_iomodule.o Modules/iobase.o Modules/fileio.o Modules/bytesio.o Modules/bufferedio.o Modules/textio.o Modules/stringio.o  Modules/zipimport.o  Modules/faulthandler.o  Modules/symtablemodule.o  Modules/xxsubtype.o -lpthread -ldl  -lutil   -lm  
      Modules/_freeze_importlib.o: In function `main':
      /home/antoine/cpython/default/Modules/_freeze_importlib.c:37: undefined reference to `PyImport_FrozenModules'
      Python/import.o: In function `find_frozen':
      /home/antoine/cpython/default/Python/import.c:978: undefined reference to `PyImport_FrozenModules'
      collect2: ld a retourné 1 code d'état d'exécution
      make[1]: *** [Modules/_freeze_importlib] Erreur 1
      make[1] : on quitte le répertoire « /home/antoine/cpython/default »
      make: *** [Python/importlib.h] Erreur 2
    12. python-dev commented on Jul 28, 2012

      python-devmannequin
      Mannequin

      New changeset 7967cb63e50e by Martin v. Löwis in branch 'default':
      Issue bpo-15431: Declare PyImport_FrozenModules conditionally on Unix only.
      http://hg.python.org/cpython/rev/7967cb63e50e

    13. python-dev commented on Jul 28, 2012

      python-devmannequin
      Mannequin

      New changeset deb421baf671 by Martin v. Löwis in branch 'default':
      Issue bpo-15431: Drop _freeze_importlib from all build configurations,
      http://hg.python.org/cpython/rev/deb421baf671

    14. loewis commented on Jul 28, 2012

      loewismannequin
      Mannequin

      The last commits should address Amaury's concerns. _freeze_importlib must now be built manually, followed by a build of pythoncore.

      Interestingly, the builds failed only once. When rebuilding, the builds succeeded as no attempt to build freeze_importlib was made. After the last change to freeze_importlib, they again built it, and that seems to work now.

      Tentatively closing the issue again.

    15. transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions