[Tensorflow] tensorflow.python.framework.errors_impl.FailedPreconditionError: Could not find variable 52/kernel. This could mean that the variable has been deleted. In TF1, it can also mean the variable is uninitialized.

Version: LPOT 1.5, Tensorflow 2.5, Intel-Tensorflow 2.5 Env: Google Colab

I was using a Keras saved model for quantization, and the following error occurs:

2021-07-23 03:49:12 [WARNING] There is no quantizable op type!!!
2021-07-23 03:49:12 [INFO] Getting FP32 model baseline...
2021-07-23 03:49:12 [INFO] Start to evaluate Tensorflow model...
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1375, in _do_call
    return fn(*args)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1360, in _run_fn
    target_list, run_metadata)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1453, in _call_tf_sessionrun
    run_metadata)
tensorflow.python.framework.errors_impl.FailedPreconditionError: Could not find variable 52/kernel. This could mean that the variable has been deleted. In TF1, it can also mean the variable is uninitialized. Debug info: container=localhost, status=Not found: Container localhost does not exist. (Could not find resource: localhost/52/kernel)
	 [[{{node model/52/Conv2D/ReadVariableOp}}]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "main.py", line 110, in <module>
    evaluate_opt_graph.run()
  File "main.py", line 94, in run
    q_model = quantizer()
  File "/usr/local/lib/python3.7/dist-packages/lpot/experimental/quantization.py", line 177, in __call__
    self.strategy.traverse()
  File "/usr/local/lib/python3.7/dist-packages/lpot/strategy/strategy.py", line 286, in traverse
    self.baseline = self._evaluate(self.model)
  File "/usr/local/lib/python3.7/dist-packages/lpot/strategy/strategy.py", line 424, in _evaluate
    val = self.objective.evaluate(eval_func, model)
  File "/usr/local/lib/python3.7/dist-packages/lpot/objective.py", line 213, in evaluate
    acc = eval_func(model)
  File "/usr/local/lib/python3.7/dist-packages/lpot/utils/create_obj_from_config.py", line 131, in eval_func
    tensorboard, fp32_baseline)
  File "/usr/local/lib/python3.7/dist-packages/lpot/adaptor/tensorflow.py", line 210, in evaluate
    predictions = model.sess.run(output_tensor, feed_dict)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 968, in run
    run_metadata_ptr)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1191, in _run
    feed_dict_tensor, options, run_metadata)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1369, in _do_run
    run_metadata)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1394, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.FailedPreconditionError: Could not find variable 52/kernel. This could mean that the variable has been deleted. In TF1, it can also mean the variable is uninitialized. Debug info: container=localhost, status=Not found: Container localhost does not exist. (Could not find resource: localhost/52/kernel)
	 [[{{node model/52/Conv2D/ReadVariableOp}}]]

Also, I don’t know why the system prints [WARNING] There is no quantizable op type!!!, because my model contains Conv2D operations and Matmul operations, which are clearly quantizable.

[peiwenhuang27]

1. Yes, I did load my model with my own script for inference, which runs smoothly and correctly outputs the MSE loss, here is a snippet of the code:

with tf.compat.v1.Session() as sess:
    tf.compat.v1.saved_model.loader.load(sess, ['serve'], model_path)
    output = sess.graph.get_tensor_by_name('output:0')
    predictions = sess.run(output, {'first_input:0': x1[:64], 'second_input:0': x2[:64]})
    mse = tf.reduce_mean(tf.keras.losses.mean_squared_error(y[:64], predictions))
    print(mse.eval())

1. As for the version of Tensorflow & Intel-Tensorflow, both are 2.5.0:

2. My subgraph structure looks somewhat like below: (Here I used ONNX model converted from the Keras saved model for its better readability, but they are of the same structure essentially)

Is “Conv2D + LeakyRelu” a supported pattern in Tensorflow? On the other hand, LPOT recognizeds Matmul in my model as quantizable, but when quantizaing it always prints Unknown Matmul.

[chensuyue]

No feedback over 2 weeks, closed at first. Please reopen if issue still there.