Skip to content

Failure to build with address sanitizer #80537

Closed

Description

@btharper
mannequin
BPO 36356
Nosy @vstinner, @ned-deily, @matrixise, @miss-islington, @btharper
PRs
  • bpo-36356: Fix leak of locale string #12425
  • bpo-36356: Release Unicode interned strings on Valgrind #12431
  • bpo-36333, bpo-36356: Fix _PyEval_FiniThreads() #12432
  • bpo-36356: pymain_free() calls _PyRuntime_Finalize() #12435
  • [3.7] bpo-36356: pymain_free() calls _PyRuntime_Finalize() #12436
  • bpo-36356: Fix leak when updating search path in config #12444
  • bpo-36356: Destroy the GIL at exit #12453
  • bpo-36356: Fix _PyCoreConfig_Read() #12454
  • bpo-36356: pymain_exit_error() only call pymain_free() for exit #12968
  • Revert "bpo-36356: Destroy the GIL at exit (GH-12453)" #13006
  • bpo-36356: Fix memory leak in _asynciomodule.c during "setup.py build_ext" #16598
  • [3.8] bpo-36356: Fix memory leak in _asynciomodule.c (GH-16598) #16622
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2019-10-08.13:03:16.055>
    created_at = <Date 2019-03-19.03:47:57.667>
    labels = ['build', '3.8']
    title = 'Failure to build with address sanitizer'
    updated_at = <Date 2019-10-08.13:03:16.054>
    user = 'https://github.com/btharper'

    bugs.python.org fields:

    activity = <Date 2019-10-08.13:03:16.054>
    actor = 'vstinner'
    assignee = 'none'
    closed = True
    closed_date = <Date 2019-10-08.13:03:16.055>
    closer = 'vstinner'
    components = ['Build']
    creation = <Date 2019-03-19.03:47:57.667>
    creator = 'btharper'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 36356
    keywords = ['patch']
    message_count = 28.0
    messages = ['338315', '338326', '338329', '338338', '338341', '338352', '338353', '338357', '338358', '338364', '338378', '338431', '338774', '338776', '338907', '340907', '340912', '341060', '341061', '341064', '341065', '341066', '354107', '354116', '354119', '354123', '354200', '354201']
    nosy_count = 5.0
    nosy_names = ['vstinner', 'ned.deily', 'matrixise', 'miss-islington', 'btharper']
    pr_nums = ['12425', '12431', '12432', '12435', '12436', '12444', '12453', '12454', '12968', '13006', '16598', '16622']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'compile error'
    url = 'https://bugs.python.org/issue36356'
    versions = ['Python 3.8']

    Activity

    1. btharper commented on Mar 19, 2019

      btharpermannequin
      MannequinAuthor

      Trying to run make after './configure --with-address-sanitizer --with-pydebug' fails with leak of locale string

    2. matrixise commented on Mar 19, 2019

      @matrixise
      Member

      just for me, btharper, which system do you use? because I have a fedora 29 and when I try to compile with these flags, getaddrinfo is not found.

    3. matrixise commented on Mar 19, 2019

      @matrixise
      Member

      ok, found

      I have to disable ipv6 and install libasan

      ./python -E -S -m sysconfig --generate-posix-vars ;\
      if test $? -ne 0 ; then \
      echo "generate-posix-vars failed" ; \
      rm -f ./pybuilddir.txt ; \
      exit 1 ; \
      fi

      =================================================================
      ==5408==ERROR: LeakSanitizer: detected memory leaks

      Direct leak of 34 byte(s) in 1 object(s) allocated from:
      #0 0x7fb085e03c08 in __interceptor_malloc (/lib64/libasan.so.5+0xefc08)
      #1 0x4d99a2 in _PyMem_RawMalloc Objects/obmalloc.c:99
      #2 0x4d9b4a in _PyMem_DebugRawAlloc Objects/obmalloc.c:2020
      #3 0x4d9ba5 in _PyMem_DebugRawMalloc Objects/obmalloc.c:2049
      #4 0x4dc354 in PyMem_RawMalloc Objects/obmalloc.c:527
      #5 0x4dc510 in _PyMem_RawStrdup Objects/obmalloc.c:613
      #6 0x652f5f in get_ctype_locale Python/preconfig.c:483
      #7 0x6560a6 in _PyPreConfig_Read Python/preconfig.c:506
      #8 0x616cde in _PyCoreConfig_ReadPreConfig Python/coreconfig.c:1342
      #9 0x620564 in _PyCoreConfig_Read Python/coreconfig.c:1378
      #10 0x65ef06 in pyinit_coreconfig Python/pylifecycle.c:740
      #11 0x65fa05 in _Py_InitializeCore Python/pylifecycle.c:786
      #12 0x42abba in pymain_init Modules/main.c:370
      #13 0x42addc in pymain_main Modules/main.c:889
      #14 0x42b2bb in _Py_UnixMain Modules/main.c:940
      #15 0x4258ee in main Programs/python.c:16
      #16 0x7fb085983412 in __libc_start_main ../csu/libc-start.c:308

      Direct leak of 34 byte(s) in 1 object(s) allocated from:
      #0 0x7fb085e03c08 in __interceptor_malloc (/lib64/libasan.so.5+0xefc08)
      #1 0x4d99a2 in _PyMem_RawMalloc Objects/obmalloc.c:99
      #2 0x4d9b4a in _PyMem_DebugRawAlloc Objects/obmalloc.c:2020
      #3 0x4d9ba5 in _PyMem_DebugRawMalloc Objects/obmalloc.c:2049
      #4 0x4dc354 in PyMem_RawMalloc Objects/obmalloc.c:527
      #5 0x4dc510 in _PyMem_RawStrdup Objects/obmalloc.c:613
      #6 0x652f5f in get_ctype_locale Python/preconfig.c:483
      #7 0x6560a6 in _PyPreConfig_Read Python/preconfig.c:506
      #8 0x6585f8 in pyinit_preconfig Python/pylifecycle.c:723
      #9 0x65f8b0 in _Py_InitializeCore Python/pylifecycle.c:781
      #10 0x42abba in pymain_init Modules/main.c:370
      #11 0x42addc in pymain_main Modules/main.c:889
      #12 0x42b2bb in _Py_UnixMain Modules/main.c:940
      #13 0x4258ee in main Programs/python.c:16
      #14 0x7fb085983412 in __libc_start_main ../csu/libc-start.c:308

      SUMMARY: AddressSanitizer: 68 byte(s) leaked in 2 allocation(s).
      generate-posix-vars failed
      make: *** [Makefile:586: pybuilddir.txt] Error 1

    4. matrixise commented on Mar 19, 2019

      @matrixise
      Member

      your PR seems to be fine.

      but I continue to get two refleaks with valgrind but don't worry, @vstinner is working on these refleaks.

      ==5440== 64 bytes in 1 blocks are possibly lost in loss record 1 of 2
      ==5440== at 0x483880B: malloc (vg_replace_malloc.c:309)
      ==5440== by 0x46BFAB: _PyMem_RawMalloc (obmalloc.c:99)
      ==5440== by 0x46BEA3: _PyMem_DebugRawAlloc (obmalloc.c:2020)
      ==5440== by 0x46BF33: _PyMem_DebugRawMalloc (obmalloc.c:2049)
      ==5440== by 0x46D304: PyMem_RawMalloc (obmalloc.c:527)
      ==5440== by 0x52C1EF: PyThread_allocate_lock (thread_pthread.h:330)
      ==5440== by 0x5154F0: _PyRuntimeState_Init_impl (pystate.c:52)
      ==5440== by 0x515A76: _PyRuntimeState_Init (pystate.c:77)
      ==5440== by 0x514114: _PyRuntime_Initialize (pylifecycle.c:88)
      ==5440== by 0x4236FE: pymain_init (main.c:339)
      ==5440== by 0x423980: pymain_main (main.c:889)
      ==5440== by 0x423A6A: _Py_UnixMain (main.c:940)
      ==5440==
      ==5440== 64 bytes in 1 blocks are possibly lost in loss record 2 of 2
      ==5440== at 0x483880B: malloc (vg_replace_malloc.c:309)
      ==5440== by 0x46BFAB: _PyMem_RawMalloc (obmalloc.c:99)
      ==5440== by 0x46BEA3: _PyMem_DebugRawAlloc (obmalloc.c:2020)
      ==5440== by 0x46BF33: _PyMem_DebugRawMalloc (obmalloc.c:2049)
      ==5440== by 0x46D304: PyMem_RawMalloc (obmalloc.c:527)
      ==5440== by 0x52C1EF: PyThread_allocate_lock (thread_pthread.h:330)
      ==5440== by 0x515506: _PyRuntimeState_Init_impl (pystate.c:58)
      ==5440== by 0x515A76: _PyRuntimeState_Init (pystate.c:77)
      ==5440== by 0x514114: _PyRuntime_Initialize (pylifecycle.c:88)
      ==5440== by 0x4236FE: pymain_init (main.c:339)
      ==5440== by 0x423980: pymain_main (main.c:889)
      ==5440== by 0x423A6A: _Py_UnixMain (main.c:940)

    5. vstinner commented on Mar 19, 2019

      @vstinner
      Member

      New changeset e130a07 by Victor Stinner (btharper) in branch 'master':
      bpo-36356: Fix memory leak in _PyPreConfig_Read() (GH-12425)
      e130a07

    6. vstinner commented on Mar 19, 2019

      @vstinner
      Member

      New changeset a712679 by Victor Stinner in branch 'master':
      bpo-36333, bpo-36356: Fix _PyEval_FiniThreads() (GH-12432)
      a712679

    7. vstinner commented on Mar 19, 2019

      @vstinner
      Member

      New changeset fecc4f2 by Victor Stinner in branch 'master':
      bpo-36356: Release Unicode interned strings on Valgrind (bpo-12431)
      fecc4f2

    8. vstinner commented on Mar 19, 2019

      @vstinner
      Member

      New changeset f5f336a by Victor Stinner in branch 'master':
      bpo-36356: pymain_free() calls _PyRuntime_Finalize() (GH-12435)
      f5f336a

    9. vstinner commented on Mar 19, 2019

      @vstinner
      Member

      New changeset 935250d by Victor Stinner in branch '3.7':
      bpo-36356: pymain_free() calls _PyRuntime_Finalize() (GH-12436)
      935250d

    10. vstinner commented on Mar 19, 2019

      @vstinner
      Member

      At least, "./python -V" no longer leaks at commit dcf6171.

      $ ./configure --with-valgrind
      $ make
      $ valgrind ./python -V
      ==9553== Memcheck, a memory error detector
      ==9553== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
      ==9553== Using Valgrind-3.14.0 and LibVEX; rerun with -h for copyright info
      ==9553== Command: ./python -V
      ==9553== 
      Python 3.8.0a2+
      ==9553== 
      ==9553== HEAP SUMMARY:
      ==9553==     in use at exit: 0 bytes in 0 blocks
      ==9553==   total heap usage: 33 allocs, 33 frees, 5,933 bytes allocated
      ==9553== 
      ==9553== All heap blocks were freed -- no leaks are possible
      ==9553== 
      ==9553== For counts of detected and suppressed errors, rerun with: -v
      ==9553== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
    11. btharper commented on Mar 19, 2019

      btharpermannequin
      MannequinAuthor

      I'm on Ubuntu 18.10/amd64 compiling with Ubuntu's GCC 8.2.0, I know there's some libraries that are missing dependencies (including bz2 and sqlite) so I may have missed the ipv6 dependencies as well.

      My eventual goal was to be able to build a pgo optimized build of cpython with the address sanitizer turned on. Currently on my build of f5f336a I see 12 failing tests and a heap use after free as part of the ctypes test suite.

      The ctypes use after free has already been submitted as bpo-36253 since it was the only thing asan caught during tests with the leak sanitizer turned off.

    12. vstinner commented on Mar 20, 2019

      @vstinner
      Member

      New changeset 4a1468e by Victor Stinner in branch 'master':
      bpo-36356: Fix _PyCoreConfig_Read() (GH-12454)
      4a1468e

    13. 5 remaining items

    14. vstinner commented on Apr 29, 2019

      @vstinner
      Member

      New changeset b36e5d6 by Victor Stinner in branch 'master':
      bpo-36356: Destroy the GIL at exit (GH-12453)
      b36e5d6

    15. vstinner commented on Apr 29, 2019

      @vstinner
      Member

      The initial issue has been fixed, so I close the issue. I prefer to get more specific issues like bpo-36724 "Clear _PyRuntime at exit" to following changes.

      Thanks Ben Harper for the bug report and the fix.

      Note: Ben, I like your songs! Good to know that you chose to contribute to Python in your free time! ;-)

    16. vstinner commented on Apr 29, 2019

      @vstinner
      Member

      Oh. test_daemon_threads_shutdown_stdout_deadlock of test_io fails randomly on buildbots.

      I ran "./python -m test test_io -m '*daemon*' -F -j4" for 5 minutes on my Fedora 29 before merging my PR, but I didn't get any failure :-(

      ======================================================================
      FAIL: test_daemon_threads_shutdown_stdout_deadlock (test.test_io.CMiscIOTest)
      ----------------------------------------------------------------------

      Traceback (most recent call last):
        File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z/build/Lib/test/test_io.py", line 4186, in test_daemon_threads_shutdown_stdout_deadlock
          self.check_daemon_threads_shutdown_deadlock('stdout')
        File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z/build/Lib/test/test_io.py", line 4177, in check_daemon_threads_shutdown_deadlock
          self.assertIn("Fatal Python error: could not acquire lock "
      AssertionError: "Fatal Python error: could not acquire lock for <_io.BufferedWriter name='<stdout>'> at interpreter shutdown, possibly due to daemon threads" not found in 'Fatal Python error: PyCOND_FINI(_PyRuntime.ceval.gil.cond) failed'

      Failures:

      AMD64 Fedora Rawhide Clang Installed 3.x: fail then pass
      https://buildbot.python.org/all/#/builders/188/builds/93

      s390x Debian 3.x: fail then pass
      https://buildbot.python.org/all/#/builders/13/builds/2799

      s390x RHEL 3.x: fail twice
      https://buildbot.python.org/all/#/builders/21/builds/2780

      AMD64 Debian root 3.x: fail twice
      https://buildbot.python.org/all/#/builders/27/builds/2759

    17. vstinner commented on Apr 29, 2019

      @vstinner
      Member

      New changeset 99fcc61 by Victor Stinner in branch 'master':
      Revert "bpo-36356: Destroy the GIL at exit (GH-12453)" (GH613006)
      99fcc61

    18. vstinner commented on Apr 29, 2019

      @vstinner
      Member

      I just reverted the change. I'm unable to reproduce and I am not available this week to investigate anyway.

    19. 1st1 commented on Oct 7, 2019

      @1st1
      Member

      New changeset 321def8 by Yury Selivanov (Ben Harper) in branch 'master':
      bpo-36356: Fix memory leak in _asynciomodule.c (GH-16598)
      321def8

    20. miss-islington commented on Oct 7, 2019

      @miss-islington
      Contributor

      New changeset 13915a3 by Miss Islington (bot) in branch '3.8':
      bpo-36356: Fix memory leak in _asynciomodule.c (GH-16598)
      13915a3

    21. vstinner commented on Oct 7, 2019

      @vstinner
      Member

      Can we close this issue, or is there a remaining known bug?

    22. ned-deily commented on Oct 7, 2019

      @ned-deily
      Member

      FWIW, the assertion error on macOS reported above in msg338774 does not seem to occur anymore with current HEAD of master and with recent system clang (Apple clang version 11.0.0 (clang-1100.0.33.8)) on macOS 10.14.6.

    23. btharper commented on Oct 8, 2019

      btharpermannequin
      MannequinAuthor

      The default build of python with ASAN builds successfully. Test suite and PGO build are currently blocked, but that can be separated if it makes more sense to handle them individually.

      Would it make sense to add an ASAN build to the CI pipeline to make detection automatic going forward?

    24. vstinner commented on Oct 8, 2019

      @vstinner
      Member

      The default build of python with ASAN builds successfully. Test suite and PGO build are currently blocked, but that can be separated if it makes more sense to handle them individually.

      Yeah, please open one issue per ASAN issue (try to group similar warnings in the same Python issue?).

      Would it make sense to add an ASAN build to the CI pipeline to make detection automatic going forward?

      Gregory maintains a clang UBSan buildbot:
      https://buildbot.python.org/all/#/builders/135

      Usually I prefer to setup a buildbot when most issues are fixed.

      I close this issue.

    25. transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      3.8 (EOL)end of lifebuildThe build process and cross-build

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions