You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee=Noneclosed_at=<Date2019-10-08.13:03:16.055>created_at=<Date2019-03-19.03:47:57.667>labels= ['build', '3.8']
title='Failure to build with address sanitizer'updated_at=<Date2019-10-08.13:03:16.054>user='https://github.com/btharper'
but I continue to get two refleaks with valgrind but don't worry, @vstinner is working on these refleaks.
==5440== 64 bytes in 1 blocks are possibly lost in loss record 1 of 2
==5440== at 0x483880B: malloc (vg_replace_malloc.c:309)
==5440== by 0x46BFAB: _PyMem_RawMalloc (obmalloc.c:99)
==5440== by 0x46BEA3: _PyMem_DebugRawAlloc (obmalloc.c:2020)
==5440== by 0x46BF33: _PyMem_DebugRawMalloc (obmalloc.c:2049)
==5440== by 0x46D304: PyMem_RawMalloc (obmalloc.c:527)
==5440== by 0x52C1EF: PyThread_allocate_lock (thread_pthread.h:330)
==5440== by 0x5154F0: _PyRuntimeState_Init_impl (pystate.c:52)
==5440== by 0x515A76: _PyRuntimeState_Init (pystate.c:77)
==5440== by 0x514114: _PyRuntime_Initialize (pylifecycle.c:88)
==5440== by 0x4236FE: pymain_init (main.c:339)
==5440== by 0x423980: pymain_main (main.c:889)
==5440== by 0x423A6A: _Py_UnixMain (main.c:940)
==5440==
==5440== 64 bytes in 1 blocks are possibly lost in loss record 2 of 2
==5440== at 0x483880B: malloc (vg_replace_malloc.c:309)
==5440== by 0x46BFAB: _PyMem_RawMalloc (obmalloc.c:99)
==5440== by 0x46BEA3: _PyMem_DebugRawAlloc (obmalloc.c:2020)
==5440== by 0x46BF33: _PyMem_DebugRawMalloc (obmalloc.c:2049)
==5440== by 0x46D304: PyMem_RawMalloc (obmalloc.c:527)
==5440== by 0x52C1EF: PyThread_allocate_lock (thread_pthread.h:330)
==5440== by 0x515506: _PyRuntimeState_Init_impl (pystate.c:58)
==5440== by 0x515A76: _PyRuntimeState_Init (pystate.c:77)
==5440== by 0x514114: _PyRuntime_Initialize (pylifecycle.c:88)
==5440== by 0x4236FE: pymain_init (main.c:339)
==5440== by 0x423980: pymain_main (main.c:889)
==5440== by 0x423A6A: _Py_UnixMain (main.c:940)
At least, "./python -V" no longer leaks at commit dcf6171.
$ ./configure --with-valgrind
$ make
$ valgrind ./python -V
==9553== Memcheck, a memory error detector
==9553== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.==9553== Using Valgrind-3.14.0 and LibVEX; rerun with -h for copyright info==9553== Command: ./python -V==9553== Python 3.8.0a2+==9553== ==9553== HEAP SUMMARY:==9553== in use at exit: 0 bytes in 0 blocks==9553== total heap usage: 33 allocs, 33 frees, 5,933 bytes allocated==9553== ==9553== All heap blocks were freed -- no leaks are possible==9553== ==9553== For counts of detected and suppressed errors, rerun with: -v==9553== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
btharper commented on Mar 19, 2019
btharpermannequin
MannequinAuthor
More actions
I'm on Ubuntu 18.10/amd64 compiling with Ubuntu's GCC 8.2.0, I know there's some libraries that are missing dependencies (including bz2 and sqlite) so I may have missed the ipv6 dependencies as well.
My eventual goal was to be able to build a pgo optimized build of cpython with the address sanitizer turned on. Currently on my build of f5f336a I see 12 failing tests and a heap use after free as part of the ctypes test suite.
The ctypes use after free has already been submitted as bpo-36253 since it was the only thing asan caught during tests with the leak sanitizer turned off.
The initial issue has been fixed, so I close the issue. I prefer to get more specific issues like bpo-36724 "Clear _PyRuntime at exit" to following changes.
Thanks Ben Harper for the bug report and the fix.
Note: Ben, I like your songs! Good to know that you chose to contribute to Python in your free time! ;-)
Traceback (most recent call last):
File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z/build/Lib/test/test_io.py", line 4186, in test_daemon_threads_shutdown_stdout_deadlockself.check_daemon_threads_shutdown_deadlock('stdout')
File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z/build/Lib/test/test_io.py", line 4177, in check_daemon_threads_shutdown_deadlockself.assertIn("Fatal Python error: could not acquire lock "AssertionError: "Fatal Python error: could not acquire lock for <_io.BufferedWriter name='<stdout>'> at interpreter shutdown, possibly due to daemon threads" not found in 'Fatal Python error: PyCOND_FINI(_PyRuntime.ceval.gil.cond) failed'
FWIW, the assertion error on macOS reported above in msg338774 does not seem to occur anymore with current HEAD of master and with recent system clang (Apple clang version 11.0.0 (clang-1100.0.33.8)) on macOS 10.14.6.
btharper commented on Oct 8, 2019
btharpermannequin
MannequinAuthor
More actions
The default build of python with ASAN builds successfully. Test suite and PGO build are currently blocked, but that can be separated if it makes more sense to handle them individually.
Would it make sense to add an ASAN build to the CI pipeline to make detection automatic going forward?
The default build of python with ASAN builds successfully. Test suite and PGO build are currently blocked, but that can be separated if it makes more sense to handle them individually.
Yeah, please open one issue per ASAN issue (try to group similar warnings in the same Python issue?).
Would it make sense to add an ASAN build to the CI pipeline to make detection automatic going forward?
Activity
btharper commentedon Mar 19, 2019
Trying to run make after './configure --with-address-sanitizer --with-pydebug' fails with leak of locale string
matrixise commentedon Mar 19, 2019
just for me, btharper, which system do you use? because I have a fedora 29 and when I try to compile with these flags, getaddrinfo is not found.
matrixise commentedon Mar 19, 2019
ok, found
I have to disable ipv6 and install libasan
./python -E -S -m sysconfig --generate-posix-vars ;\
if test $? -ne 0 ; then \
echo "generate-posix-vars failed" ; \
rm -f ./pybuilddir.txt ; \
exit 1 ; \
fi
=================================================================
==5408==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 34 byte(s) in 1 object(s) allocated from:
#0 0x7fb085e03c08 in __interceptor_malloc (/lib64/libasan.so.5+0xefc08)
#1 0x4d99a2 in _PyMem_RawMalloc Objects/obmalloc.c:99
#2 0x4d9b4a in _PyMem_DebugRawAlloc Objects/obmalloc.c:2020
#3 0x4d9ba5 in _PyMem_DebugRawMalloc Objects/obmalloc.c:2049
#4 0x4dc354 in PyMem_RawMalloc Objects/obmalloc.c:527
#5 0x4dc510 in _PyMem_RawStrdup Objects/obmalloc.c:613
#6 0x652f5f in get_ctype_locale Python/preconfig.c:483
#7 0x6560a6 in _PyPreConfig_Read Python/preconfig.c:506
#8 0x616cde in _PyCoreConfig_ReadPreConfig Python/coreconfig.c:1342
#9 0x620564 in _PyCoreConfig_Read Python/coreconfig.c:1378
#10 0x65ef06 in pyinit_coreconfig Python/pylifecycle.c:740
#11 0x65fa05 in _Py_InitializeCore Python/pylifecycle.c:786
#12 0x42abba in pymain_init Modules/main.c:370
#13 0x42addc in pymain_main Modules/main.c:889
#14 0x42b2bb in _Py_UnixMain Modules/main.c:940
#15 0x4258ee in main Programs/python.c:16
#16 0x7fb085983412 in __libc_start_main ../csu/libc-start.c:308
Direct leak of 34 byte(s) in 1 object(s) allocated from:
#0 0x7fb085e03c08 in __interceptor_malloc (/lib64/libasan.so.5+0xefc08)
#1 0x4d99a2 in _PyMem_RawMalloc Objects/obmalloc.c:99
#2 0x4d9b4a in _PyMem_DebugRawAlloc Objects/obmalloc.c:2020
#3 0x4d9ba5 in _PyMem_DebugRawMalloc Objects/obmalloc.c:2049
#4 0x4dc354 in PyMem_RawMalloc Objects/obmalloc.c:527
#5 0x4dc510 in _PyMem_RawStrdup Objects/obmalloc.c:613
#6 0x652f5f in get_ctype_locale Python/preconfig.c:483
#7 0x6560a6 in _PyPreConfig_Read Python/preconfig.c:506
#8 0x6585f8 in pyinit_preconfig Python/pylifecycle.c:723
#9 0x65f8b0 in _Py_InitializeCore Python/pylifecycle.c:781
#10 0x42abba in pymain_init Modules/main.c:370
#11 0x42addc in pymain_main Modules/main.c:889
#12 0x42b2bb in _Py_UnixMain Modules/main.c:940
#13 0x4258ee in main Programs/python.c:16
#14 0x7fb085983412 in __libc_start_main ../csu/libc-start.c:308
SUMMARY: AddressSanitizer: 68 byte(s) leaked in 2 allocation(s).
generate-posix-vars failed
make: *** [Makefile:586: pybuilddir.txt] Error 1
matrixise commentedon Mar 19, 2019
your PR seems to be fine.
but I continue to get two refleaks with valgrind but don't worry, @vstinner is working on these refleaks.
==5440== 64 bytes in 1 blocks are possibly lost in loss record 1 of 2
==5440== at 0x483880B: malloc (vg_replace_malloc.c:309)
==5440== by 0x46BFAB: _PyMem_RawMalloc (obmalloc.c:99)
==5440== by 0x46BEA3: _PyMem_DebugRawAlloc (obmalloc.c:2020)
==5440== by 0x46BF33: _PyMem_DebugRawMalloc (obmalloc.c:2049)
==5440== by 0x46D304: PyMem_RawMalloc (obmalloc.c:527)
==5440== by 0x52C1EF: PyThread_allocate_lock (thread_pthread.h:330)
==5440== by 0x5154F0: _PyRuntimeState_Init_impl (pystate.c:52)
==5440== by 0x515A76: _PyRuntimeState_Init (pystate.c:77)
==5440== by 0x514114: _PyRuntime_Initialize (pylifecycle.c:88)
==5440== by 0x4236FE: pymain_init (main.c:339)
==5440== by 0x423980: pymain_main (main.c:889)
==5440== by 0x423A6A: _Py_UnixMain (main.c:940)
==5440==
==5440== 64 bytes in 1 blocks are possibly lost in loss record 2 of 2
==5440== at 0x483880B: malloc (vg_replace_malloc.c:309)
==5440== by 0x46BFAB: _PyMem_RawMalloc (obmalloc.c:99)
==5440== by 0x46BEA3: _PyMem_DebugRawAlloc (obmalloc.c:2020)
==5440== by 0x46BF33: _PyMem_DebugRawMalloc (obmalloc.c:2049)
==5440== by 0x46D304: PyMem_RawMalloc (obmalloc.c:527)
==5440== by 0x52C1EF: PyThread_allocate_lock (thread_pthread.h:330)
==5440== by 0x515506: _PyRuntimeState_Init_impl (pystate.c:58)
==5440== by 0x515A76: _PyRuntimeState_Init (pystate.c:77)
==5440== by 0x514114: _PyRuntime_Initialize (pylifecycle.c:88)
==5440== by 0x4236FE: pymain_init (main.c:339)
==5440== by 0x423980: pymain_main (main.c:889)
==5440== by 0x423A6A: _Py_UnixMain (main.c:940)
vstinner commentedon Mar 19, 2019
New changeset e130a07 by Victor Stinner (btharper) in branch 'master':
bpo-36356: Fix memory leak in _PyPreConfig_Read() (GH-12425)
e130a07
vstinner commentedon Mar 19, 2019
New changeset a712679 by Victor Stinner in branch 'master':
bpo-36333, bpo-36356: Fix _PyEval_FiniThreads() (GH-12432)
a712679
vstinner commentedon Mar 19, 2019
New changeset fecc4f2 by Victor Stinner in branch 'master':
bpo-36356: Release Unicode interned strings on Valgrind (bpo-12431)
fecc4f2
vstinner commentedon Mar 19, 2019
New changeset f5f336a by Victor Stinner in branch 'master':
bpo-36356: pymain_free() calls _PyRuntime_Finalize() (GH-12435)
f5f336a
vstinner commentedon Mar 19, 2019
New changeset 935250d by Victor Stinner in branch '3.7':
bpo-36356: pymain_free() calls _PyRuntime_Finalize() (GH-12436)
935250d
vstinner commentedon Mar 19, 2019
At least, "./python -V" no longer leaks at commit dcf6171.
btharper commentedon Mar 19, 2019
I'm on Ubuntu 18.10/amd64 compiling with Ubuntu's GCC 8.2.0, I know there's some libraries that are missing dependencies (including bz2 and sqlite) so I may have missed the ipv6 dependencies as well.
My eventual goal was to be able to build a pgo optimized build of cpython with the address sanitizer turned on. Currently on my build of f5f336a I see 12 failing tests and a heap use after free as part of the ctypes test suite.
The ctypes use after free has already been submitted as bpo-36253 since it was the only thing asan caught during tests with the leak sanitizer turned off.
vstinner commentedon Mar 20, 2019
New changeset 4a1468e by Victor Stinner in branch 'master':
bpo-36356: Fix _PyCoreConfig_Read() (GH-12454)
4a1468e
5 remaining items
vstinner commentedon Apr 29, 2019
New changeset b36e5d6 by Victor Stinner in branch 'master':
bpo-36356: Destroy the GIL at exit (GH-12453)
b36e5d6
vstinner commentedon Apr 29, 2019
The initial issue has been fixed, so I close the issue. I prefer to get more specific issues like bpo-36724 "Clear _PyRuntime at exit" to following changes.
Thanks Ben Harper for the bug report and the fix.
Note: Ben, I like your songs! Good to know that you chose to contribute to Python in your free time! ;-)
vstinner commentedon Apr 29, 2019
Oh. test_daemon_threads_shutdown_stdout_deadlock of test_io fails randomly on buildbots.
I ran "./python -m test test_io -m '*daemon*' -F -j4" for 5 minutes on my Fedora 29 before merging my PR, but I didn't get any failure :-(
======================================================================
FAIL: test_daemon_threads_shutdown_stdout_deadlock (test.test_io.CMiscIOTest)
----------------------------------------------------------------------
Failures:
AMD64 Fedora Rawhide Clang Installed 3.x: fail then pass
https://buildbot.python.org/all/#/builders/188/builds/93
s390x Debian 3.x: fail then pass
https://buildbot.python.org/all/#/builders/13/builds/2799
s390x RHEL 3.x: fail twice
https://buildbot.python.org/all/#/builders/21/builds/2780
AMD64 Debian root 3.x: fail twice
https://buildbot.python.org/all/#/builders/27/builds/2759
vstinner commentedon Apr 29, 2019
New changeset 99fcc61 by Victor Stinner in branch 'master':
Revert "bpo-36356: Destroy the GIL at exit (GH-12453)" (GH613006)
99fcc61
vstinner commentedon Apr 29, 2019
I just reverted the change. I'm unable to reproduce and I am not available this week to investigate anyway.
1st1 commentedon Oct 7, 2019
New changeset 321def8 by Yury Selivanov (Ben Harper) in branch 'master':
bpo-36356: Fix memory leak in _asynciomodule.c (GH-16598)
321def8
miss-islington commentedon Oct 7, 2019
New changeset 13915a3 by Miss Islington (bot) in branch '3.8':
bpo-36356: Fix memory leak in _asynciomodule.c (GH-16598)
13915a3
vstinner commentedon Oct 7, 2019
Can we close this issue, or is there a remaining known bug?
ned-deily commentedon Oct 7, 2019
FWIW, the assertion error on macOS reported above in msg338774 does not seem to occur anymore with current HEAD of master and with recent system clang (Apple clang version 11.0.0 (clang-1100.0.33.8)) on macOS 10.14.6.
btharper commentedon Oct 8, 2019
The default build of python with ASAN builds successfully. Test suite and PGO build are currently blocked, but that can be separated if it makes more sense to handle them individually.
Would it make sense to add an ASAN build to the CI pipeline to make detection automatic going forward?
vstinner commentedon Oct 8, 2019
Yeah, please open one issue per ASAN issue (try to group similar warnings in the same Python issue?).
Gregory maintains a clang UBSan buildbot:
https://buildbot.python.org/all/#/builders/135
Usually I prefer to setup a buildbot when most issues are fixed.
I close this issue.