Tensorflow Exception 0xC0000409 & Numba Error

Here I want to report an issue.

Hardware Environmet:

RAM: 8GB
No stand-alone graphic card. So I use the tensorFlow cpu version. (pip install tensorflow==1.5.0)

Problem

1. I run the following command: python segment.py ./app1.jpg

When executing this line in ai.py:

def go_vector(x):
    return session.run(vector_op, feed_dict={
        ip3: x[None, :, :, :]
    })[0]

I got the error Tensorflow Exception 0xC0000409, as the following pictures:

I choose VS2008 debugger, and an exception window appeared"System detected an overrun of a stack-based buffer in this application":

From the following debug log (python pdb), it seems that this error is caused by Tensorflow:

(DanbooRegion) D:\DanbooRegion-200804\code
python -m pdb segment.py ./app1.jpg
> d:\danbooregion-200804\code\segment.py(1
<module>()
-> from tricks import *
(Pdb) b 6
Breakpoint 1 at d:\danbooregion-200804\cod\segment.py:6
(Pdb) c
Using TensorFlow backend.
2020-10-25 11:24:17.511287: I C:\tf_jenkins\workspace\rel-win\M\windows\PY\36\tensorflow\
ore\platform\cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlo
 binary was not compiled to use: AVX
E:\Programs\VirtualEnvs\DanbooRegion\lib\site-packages\keras\engine\saving.py:292: UserWa
ning: No training configuration found in save file: the model was *not* compiled. Compile
it manually.
  warnings.warn('No training configuration found in save file: '
begin load
> d:\danbooregion-200804\code\segment.py(6
go_flipped_vector()
-> a = go_vector(x)
(Pdb) s
--Call--
> d:\danbooregion-200804\code\ai.py(69)go_ector()
-> def go_vector(x):
(Pdb) n
> d:\danbooregion-200804\code\ai.py(70)go_ector()
-> return session.run(vector_op, feed_dict={
(Pdb) n
> d:\danbooregion-200804\code\ai.py(71)go_ector()
-> ip3: x[None, :, :, :]
(Pdb) s
--Call--
> e:\programs\virtualenvs\danbooregion\lib\site-packages\tensorflow\python\framework\ops.
y(564)__hash__()
-> def __hash__(self):
(Pdb) n
> e:\programs\virtualenvs\danbooregion\lib\site-packages\tensorflow\python\framework\ops.
y(566)__hash__()
-> return id(self)
(Pdb) n
--Return--
> e:\programs\virtualenvs\danbooregion\lib\site-packages\tensorflow\python\framework\ops.
y(566)__hash__()->705670619320
-> return id(self)
(Pdb) n
--Call--
> e:\programs\virtualenvs\danbooregion\lib\site-packages\tensorflow\python\client\session
py(787)run()
-> def run(self, fetches, feed_dict=None, options=None, run_metadata=None):
(Pdb) n
> e:\programs\virtualenvs\danbooregion\lib\site-packages\tensorflow\python\client\session
py(890)run()
-> compat.as_bytes(options.SerializeToString())) if options else None
(Pdb) n
> e:\programs\virtualenvs\danbooregion\lib\site-packages\tensorflow\python\client\session
py(891)run()
-> run_metadata_ptr = tf_session.TF_NewBuffer() if run_metadata else None
(Pdb) n
> e:\programs\virtualenvs\danbooregion\lib\site-packages\tensorflow\python\client\session
py(893)run()
-> try:
(Pdb) n
> e:\programs\virtualenvs\danbooregion\lib\site-packages\tensorflow\python\client\session
py(894)run()
-> result = self._run(None, fetches, feed_dict, options_ptr,
(Pdb) n
> e:\programs\virtualenvs\danbooregion\lib\site-packages\tensorflow\python\client\session
py(895)run()
-> run_metadata_ptr)
(Pdb) n
[Then the tensorflow exception window appeared.]

[gceboh]

Solution

For Tensorflow exception

It is likely to be caused by not having enough physical memory. And reducing image resolution like this can solve this problem! (But keep in mind that reducing image resolution may reduce the segmentation quality a little.)

For Numba error

Solution 1: (Recommended) Manually specify the version of llvmlite:

pip install numba==0.39.0
pip install llvmlite==0.24.0

(Now no need to remove @njit annotations in tricks.py)

---

Solution 2: pip install numba==0.49.0

In numba-0.49.0, the bug of requirements.txt was fixed. The version of llvmlite have an upper bound in requirements.txt:

llvmlite>=0.31,<0.33

So pip will auto install llvmlite-0.32.1, even after the release of newer llvmlite version.

(Now no need to remove @njit annotations in tricks.py)

[gceboh]

In general, the result after reducing resolution is enough for me.