Running deepconsensus results in "free(): invalid pointer" error

I installed deepconsensus via pip in a virtualenv like this:

virtualenv /apps/deepconsensus/1.0.0/python-3.8.2_cpu
source /apps/deepconsensus/1.0.0/python-3.8.2_cpu/bin/activate
pip install pyyaml==5.4.1 'deepconsensus[cpu]==1.0.0'

I used pyyaml==5.4.1 since tf-models-official 2.10.0 requires pyyaml<6.0,>=5.1. I’m using Python 3.8.2.

When I run deepconsensus, even just for the help message, it fails–deepconsensus -h resulted in this error message:

2022-11-10 12:51:17.016037: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F AVX512_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
*** Error in `/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python': free(): invalid pointer: 0x00007f075c296c80 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x81329)[0x7f078b91d329]
/lib64/libstdc++.so.6(_ZNSt6locale5_Impl16_M_install_facetEPKNS_2idEPKNS_5facetE+0x142)[0x7f075c000192]
/lib64/libstdc++.so.6(_ZNSt6locale5_ImplC1Em+0x1e3)[0x7f075c0005e3]
/lib64/libstdc++.so.6(+0x71555)[0x7f075c001555]
/lib64/libpthread.so.0(+0x620b)[0x7f078c37920b]
/lib64/libstdc++.so.6(+0x715a1)[0x7f075c0015a1]
/lib64/libstdc++.so.6(_ZNSt6localeC2Ev+0x13)[0x7f075c0015e3]
/lib64/libstdc++.so.6(_ZNSt8ios_base4InitC2Ev+0xbc)[0x7f075bffe43c]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/lib/python3.8/site-packages/google/protobuf/pyext/_message.cpython-38-x86_64-linux-gnu.so(+0xb1150)[0x7f075bdd0150]
/lib64/ld-linux-x86-64.so.2(+0xf9c3)[0x7f078c7d59c3]
/lib64/ld-linux-x86-64.so.2(+0x1459e)[0x7f078c7da59e]
/lib64/ld-linux-x86-64.so.2(+0xf7d4)[0x7f078c7d57d4]
/lib64/ld-linux-x86-64.so.2(+0x13b8b)[0x7f078c7d9b8b]
/lib64/libdl.so.2(+0xfab)[0x7f078c16ffab]
/lib64/ld-linux-x86-64.so.2(+0xf7d4)[0x7f078c7d57d4]
/lib64/libdl.so.2(+0x15ad)[0x7f078c1705ad]
/lib64/libdl.so.2(dlopen+0x31)[0x7f078c170041]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python(_PyImport_FindSharedFuncptr+0x16b)[0x539abb]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python(_PyImport_LoadDynamicModuleWithSpec+0x159)[0x503e69]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python[0x501a23]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python[0x46f563]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python(PyVectorcall_Call+0x5c)[0x439d8c]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python(_PyEval_EvalFrameDefault+0x76d8)[0x42a308]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python(_PyEval_EvalCodeWithName+0xadf)[0x4e171f]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python(_PyFunction_Vectorcall+0x90)[0x438570]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python[0x422821]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python(_PyEval_EvalFrameDefault+0x5f91)[0x428bc1]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python[0x421571]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python[0x422821]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python(_PyEval_EvalFrameDefault+0x1fb5)[0x424be5]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python[0x421571]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python[0x422821]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python(_PyEval_EvalFrameDefault+0x15af)[0x4241df]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python[0x421571]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python[0x422821]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python(_PyEval_EvalFrameDefault+0x15af)[0x4241df]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python[0x421571]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python[0x422821]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python(_PyEval_EvalFrameDefault+0x15af)[0x4241df]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python[0x421571]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python[0x437f74]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python(_PyObject_CallMethodIdObjArgs+0xf1)[0x439831]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python(PyImport_ImportModuleLevelObject+0x3fd)[0x502c8d]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python[0x5ee426]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python[0x437c24]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python(_PyEval_EvalFrameDefault+0x76d8)[0x42a308]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python(_PyEval_EvalCodeWithName+0xadf)[0x4e171f]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python(_PyFunction_Vectorcall+0x90)[0x438570]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python[0x422821]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python(_PyEval_EvalFrameDefault+0x15af)[0x4241df]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python(_PyEval_EvalCodeWithName+0xadf)[0x4e171f]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python(_PyFunction_Vectorcall+0x90)[0x438570]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python[0x437f74]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python(_PyObject_CallMethodIdObjArgs+0xf1)[0x439831]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python(PyImport_ImportModuleLevelObject+0x4e6)[0x502d76]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python(_PyEval_EvalFrameDefault+0x6e78)[0x429aa8]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python(_PyEval_EvalCodeWithName+0xadf)[0x4e171f]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python(PyEval_EvalCode+0x23)[0x4e1b43]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python[0x5efe34]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python[0x46f563]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python(PyVectorcall_Call+0x5c)[0x439d8c]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python(_PyEval_EvalFrameDefault+0x76d8)[0x42a308]
/zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/bin/python(_PyEval_EvalCodeWithName+0xadf)[0x4e171f]
======= Memory map: ========
00400000-006f3000 r-xp 00000000 00:31 37327427                           /zapps7/python/3.8.2/gcc-9.2.0/bin/python3.8
008f2000-008f3000 r--p 002f2000 00:31 37327427                           /zapps7/python/3.8.2/gcc-9.2.0/bin/python3.8
008f3000-0092b000 rw-p 002f3000 00:31 37327427                           /zapps7/python/3.8.2/gcc-9.2.0/bin/python3.8
0092b000-0094c000 rw-p 00000000 00:00 0 
01dd2000-03209000 rw-p 00000000 00:00 0                                  [heap]
7f0754000000-7f0754021000 rw-p 00000000 00:00 0 
7f0754021000-7f0758000000 ---p 00000000 00:00 0 
7f075bd1f000-7f075bdbc000 r--p 00000000 00:31 77736144                   /zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/lib/python3.8/site-packages/google/protobuf/pyext/_message.cpython-38-x86_64-linux-gnu.so
7f075bdbc000-7f075bf0b000 r-xp 0009d000 00:31 77736144                   /zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/lib/python3.8/site-packages/google/protobuf/pyext/_message.cpython-38-x86_64-linux-gnu.so
7f075bf0b000-7f075bf7f000 r--p 001ec000 00:31 77736144                   /zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/lib/python3.8/site-packages/google/protobuf/pyext/_message.cpython-38-x86_64-linux-gnu.so
7f075bf7f000-7f075bf80000 ---p 00260000 00:31 77736144                   /zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/lib/python3.8/site-packages/google/protobuf/pyext/_message.cpython-38-x86_64-linux-gnu.so
7f075bf80000-7f075bf85000 r--p 00260000 00:31 77736144                   /zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/lib/python3.8/site-packages/google/protobuf/pyext/_message.cpython-38-x86_64-linux-gnu.so
7f075bf85000-7f075bf8f000 rw-p 00265000 00:31 77736144                   /zapps7/deepconsensus/1.0.0/python-3.8.2_cpu/lib/python3.8/site-packages/google/protobuf/pyext/_message.cpython-38-x86_64-linux-gnu.sozsh: abort      deepconsensus -h

Is there something different I should have done when installing?

[pichuan]

Thanks for the updates @mjg0 ! Glad that it worked. I’ll close this issue now. Feel free to open another one if you encounter any other issues.

[pichuan]

OK, one more update. Here is what my requirements.txt currently looks like. Not finalize yet because I’m still testing. But sharing in advance in case @mjg0 you want to test out on your end:

numpy==1.20.3
pandas==1.5.1
tf-models-official==2.9.2
pyyaml==5.4.1  # because of tf-models-official
ml_collections==0.1.1
absl-py==1.0.0
keras==2.9.0
pysam==0.19.0

(and setup.py should also pin:

    'cpu': ['intel-tensorflow==2.9.1'],
    'gpu': ['tensorflow-gpu==2.9.1']

)