Versions of tensorflow/CUDA/Keras confirmed to work ?

Hello again ! I’ve been able to train a model, but now i can’t evaluate it, i get this error message :

(deeptag) PS A:\NN\deeptag\deepdanbooru> deepdanbooru evaluate "A:\TEST\" --project-path "A:\NN\DeepTag\test1" --allow-folder
2021-12-05 16:44:56.034612: E tensorflow/stream_executor/cuda/cuda_driver.cc:271] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected
2021-12-05 16:44:56.040122: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:169] retrieving CUDA diagnostic information for host: DESKTOP-68LU6C2
2021-12-05 16:44:56.043203: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:176] hostname: DESKTOP-68LU6C2
2021-12-05 16:44:56.045541: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
Tags of A:\TEST\0beb247119fd16887de4a1feffe37e76.jpg:
Traceback (most recent call last):
  File "A:\ANACONDA\envs\deeptag\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "A:\ANACONDA\envs\deeptag\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "A:\ANACONDA\envs\deeptag\Scripts\deepdanbooru.exe\__main__.py", line 7, in <module>
  File "A:\ANACONDA\envs\deeptag\lib\site-packages\click\core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "A:\ANACONDA\envs\deeptag\lib\site-packages\click\core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "A:\ANACONDA\envs\deeptag\lib\site-packages\click\core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "A:\ANACONDA\envs\deeptag\lib\site-packages\click\core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "A:\ANACONDA\envs\deeptag\lib\site-packages\click\core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "A:\ANACONDA\envs\deeptag\lib\site-packages\deepdanbooru\__main__.py", line 85, in evaluate
    dd.commands.evaluate(target_paths, project_path, model_path, tags_path, threshold, allow_gpu, compile_model, allow_folder, folder_filters, verbose)
  File "A:\ANACONDA\envs\deeptag\lib\site-packages\deepdanbooru\commands\evaluate.py", line 74, in evaluate
    for tag, score in evaluate_image(image_path, model, tags, threshold):
  File "A:\ANACONDA\envs\deeptag\lib\site-packages\deepdanbooru\commands\evaluate.py", line 27, in evaluate_image
    result_dict[tag] = y[i]
IndexError: index 181 is out of bounds for axis 0 with size 181

I tried changing versions of CUDA (11.5, 11.2, 11.0, 10.2) and versions of tensorflow-gpu (2.7.0 and 2.6.0)

And the only change i’ve had was to get this error message instead of the previous one, with tensorflow-gpu 2.6.0

(deeptag) PS A:\NN\deeptag\deepdanbooru> deepdanbooru evaluate "A:\TEST\" --project-path "A:\NN\DeepTag\test1" --allow-folder
2021-12-05 16:10:20.496041: E tensorflow/core/lib/monitoring/collection_registry.cc:77] Cannot register 2 metrics with the same name: /tensorflow/api/keras/optimizers
Traceback (most recent call last):
  File "A:\ANACONDA\envs\deeptag\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "A:\ANACONDA\envs\deeptag\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "A:\ANACONDA\envs\deeptag\Scripts\deepdanbooru.exe\__main__.py", line 7, in <module>
  File "A:\ANACONDA\envs\deeptag\lib\site-packages\click\core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "A:\ANACONDA\envs\deeptag\lib\site-packages\click\core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "A:\ANACONDA\envs\deeptag\lib\site-packages\click\core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "A:\ANACONDA\envs\deeptag\lib\site-packages\click\core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "A:\ANACONDA\envs\deeptag\lib\site-packages\click\core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "A:\ANACONDA\envs\deeptag\lib\site-packages\deepdanbooru\__main__.py", line 85, in evaluate
    dd.commands.evaluate(target_paths, project_path, model_path, tags_path, threshold, allow_gpu, compile_model, allow_folder, folder_filters, verbose)
  File "A:\ANACONDA\envs\deeptag\lib\site-packages\deepdanbooru\commands\evaluate.py", line 61, in evaluate
    model = dd.project.load_model_from_project(project_path, compile_model=compile_model)
  File "A:\ANACONDA\envs\deeptag\lib\site-packages\deepdanbooru\project\project.py", line 43, in load_model_from_project
    model = tf.keras.models.load_model(model_path, compile=compile_model)
  File "A:\ANACONDA\envs\deeptag\lib\site-packages\tensorflow\python\util\lazy_loader.py", line 62, in __getattr__
    module = self._load()
  File "A:\ANACONDA\envs\deeptag\lib\site-packages\tensorflow\python\util\lazy_loader.py", line 45, in _load
    module = importlib.import_module(self.__name__)
  File "A:\ANACONDA\envs\deeptag\lib\importlib\__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "A:\ANACONDA\envs\deeptag\lib\site-packages\keras\__init__.py", line 25, in <module>
    from keras import models
  File "A:\ANACONDA\envs\deeptag\lib\site-packages\keras\models.py", line 20, in <module>
    from keras import metrics as metrics_module
  File "A:\ANACONDA\envs\deeptag\lib\site-packages\keras\metrics.py", line 26, in <module>
    from keras import activations
  File "A:\ANACONDA\envs\deeptag\lib\site-packages\keras\activations.py", line 20, in <module>
    from keras.layers import advanced_activations
  File "A:\ANACONDA\envs\deeptag\lib\site-packages\keras\layers\__init__.py", line 23, in <module>
    from keras.engine.input_layer import Input
  File "A:\ANACONDA\envs\deeptag\lib\site-packages\keras\engine\input_layer.py", line 21, in <module>
    from keras.engine import base_layer
  File "A:\ANACONDA\envs\deeptag\lib\site-packages\keras\engine\base_layer.py", line 43, in <module>
    from keras.mixed_precision import loss_scale_optimizer
  File "A:\ANACONDA\envs\deeptag\lib\site-packages\keras\mixed_precision\loss_scale_optimizer.py", line 18, in <module>
    from keras import optimizers
  File "A:\ANACONDA\envs\deeptag\lib\site-packages\keras\optimizers.py", line 26, in <module>
    from keras.optimizer_v2 import adadelta as adadelta_v2
  File "A:\ANACONDA\envs\deeptag\lib\site-packages\keras\optimizer_v2\adadelta.py", line 22, in <module>
    from keras.optimizer_v2 import optimizer_v2
  File "A:\ANACONDA\envs\deeptag\lib\site-packages\keras\optimizer_v2\optimizer_v2.py", line 36, in <module>
    keras_optimizers_gauge = tf.__internal__.monitoring.BoolGauge(
  File "A:\ANACONDA\envs\deeptag\lib\site-packages\tensorflow\python\eager\monitoring.py", line 360, in __init__
    super(BoolGauge, self).__init__('BoolGauge', _bool_gauge_methods,
  File "A:\ANACONDA\envs\deeptag\lib\site-packages\tensorflow\python\eager\monitoring.py", line 135, in __init__
    self._metric = self._metric_methods[self._label_length].create(*args)
tensorflow.python.framework.errors_impl.AlreadyExistsError: Another metric with the same name already exists.

i’m stuck, i’ve been on this 2 full days now and i don’t get it… (and my internet is super slow, uninstall/reinstall is a pain…) Do you have exact versions confirmed to work of tensorflow/cuda/cupti/cudnn and such ? or does the problem comes from somewhere else ? Thank you for your help

[DHG-Dav]

OMFG you’re right !!! when i last checked the tags i did hit a random key on a new line without noticing it and it screwed everything up ! It works now !!! Thank you, i was so dumb to didn’t even realize the number was related to my tags -_-’ Thanks so much, it’s working perfectly now ^^ let’s just hope i can train my model again after uninstalling/reinstalling so many times tensorflow-cuda-cudnn and such lol Thank you very much for you help !

[Superfloh]

That looks better already, which pre-trained model version did you take ?

EDIT: Forgot that you trained your own, since I used a pre-trained model with transfer learning. The error probably has something to do with the tags you used to train the model, as far as I remember it reads the tags.txt to map the model output to the tags. It looks like the tag file you used to train differs from the one you use to evaluate? Did you train it with 181 tags? Maybe check the model output dimension and the number of tags your tagfile has.