Issue with multiple input quantization(tf savedmodel)

Hi Team, thanks for the wonderful tool. I am trying to quantize savedmodel with multiple inputs but facing below issue.

Code:

import tensorflow as tf
from neural_compressor.experimental import Quantization, common

quantizer = Quantization()
quantizer.model = '../model/'
dataset = quantizer.dataset('dummy', shape=[(1,256),(1,256)])
quantizer.calib_dataloader = common.DataLoader(dataset)
quantized_model = quantizer.fit()
output_path = '../model/'
quantized_model.save(output_path)

2022-07-15 10:10:03 [WARNING] Force convert framework model to neural_compressor model. 2022-07-15 10:10:03.488839: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2022-07-15 10:10:03.490012: I tensorflow/core/common_runtime/process_util.cc:146] Creating new thread pool with default inter op setting: WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually. 2022-07-15 10:10:05 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually. WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually. 2022-07-15 10:10:08 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually. WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually. 2022-07-15 10:10:10 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually. 2022-07-15 10:10:10 [WARNING] Output tensor names should not be empty. 2022-07-15 10:10:10 [WARNING] Input tensor names should not be empty. 2022-07-15 10:10:10 [INFO] Generate a fake evaluation function. WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually. 2022-07-15 10:10:13 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually. 2022-07-15 10:10:13.470210: I tensorflow/core/grappler/devices.cc:75] Number of eligible GPUs (core count >= 8, compute capability >= 2022-07-15 10:10:13.470453: I tensorflow/core/grappler/clusters/single_machine.cc:358] Starting new session 2022-07-15 10:10:14.615203: I tensorflow/core/grappler/devices.cc:75] Number of eligible GPUs (core count >= 8, compute capability >= 2022-07-15 10:10:14.615386: I tensorflow/core/grappler/clusters/single_machine.cc:358] Starting new session WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually. 2022-07-15 10:10:18 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually. WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually. 2022-07-15 10:10:20 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually. 2022-07-15 10:10:20 [INFO] ConvertLayoutOptimizer elapsed time: 0.32 ms 2022-07-15 10:10:23.822868: I tensorflow/core/grappler/devices.cc:75] Number of eligible GPUs (core count >= 8, compute capability >= 2022-07-15 10:10:23.823242: I tensorflow/core/grappler/clusters/single_machine.cc:358] Starting new session 2022-07-15 10:10:24 [INFO] Pass GrapplerOptimizer elapsed time: 3781.35 ms 2022-07-15 10:10:24 [INFO] Pass SwitchOptimizer elapsed time: 277.86 ms 2022-07-15 10:10:24 [INFO] Pass RemoveTrainingNodesOptimizer elapsed time: 275.21 ms 2022-07-15 10:10:24 [INFO] Pass SplitSharedInputOptimizer elapsed time: 5.65 ms 2022-07-15 10:10:25 [INFO] Pass GraphFoldConstantOptimizer elapsed time: 276.08 ms 2022-07-15 10:10:25 [INFO] Pass FuseColumnWiseMulOptimizer elapsed time: 275.88 ms WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/util.py:322: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version. Instructions for updating: Use tf.compat.v1.graph_util.extract_sub_graph 2022-07-15 10:10:25 [WARNING] From /opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/util.py:322: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version. Instructions for updating: Use tf.compat.v1.graph_util.extract_sub_graph 2022-07-15 10:10:26 [INFO] Pass StripUnusedNodesOptimizer elapsed time: 1121.14 ms 2022-07-15 10:10:26 [INFO] Pass GraphCseOptimizer elapsed time: 279.52 ms 2022-07-15 10:10:27 [INFO] Pass FuseDecomposedBNOptimizer elapsed time: 539.8 ms 2022-07-15 10:10:27 [INFO] Pass FoldBatchNormNodesOptimizer elapsed time: 277.56 ms 2022-07-15 10:10:27 [INFO] Pass UpdateEnterOptimizer elapsed time: 277.27 ms 2022-07-15 10:10:28 [INFO] Pass ConvertLeakyReluOptimizer elapsed time: 279.11 ms 2022-07-15 10:10:28 [INFO] Pass ConvertAddToBiasAddOptimizer elapsed time: 275.88 ms 2022-07-15 10:10:28 [INFO] Pass FuseTransposeReshapeOptimizer elapsed time: 277.57 ms 2022-07-15 10:10:29 [INFO] Pass FuseConvWithMathOptimizer elapsed time: 276.92 ms 2022-07-15 10:10:29 [INFO] Pass ExpandDimsOptimizer elapsed time: 275.92 ms 2022-07-15 10:10:29 [INFO] Pass FetchWeightFromReshapeOptimizer elapsed time: 279.25 ms 2022-07-15 10:10:29 [INFO] Pass InjectDummyBiasAddOptimizer elapsed time: 282.46 ms 2022-07-15 10:10:30 [INFO] Pass MoveSqueezeAfterReluOptimizer elapsed time: 273.58 ms 2022-07-15 10:10:31 [INFO] Pass Pre Optimization elapsed time: 15803.25 ms 2022-07-15 10:10:32 [INFO] Get FP32 model baseline. 2022-07-15 10:10:32 [INFO] Save tuning history to /home/jupyter/model-compressor/conversion_notebooks/nc_workspace/2022-07-15_10-10-02/./history.snapshot. 2022-07-15 10:10:32 [INFO] FP32 baseline is: [Accuracy: 1.0000, Duration (seconds): 0.0000] WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually. 2022-07-15 10:10:34 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually. WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually. 2022-07-15 10:10:36 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually. WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually. 2022-07-15 10:10:39 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually. WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually. 2022-07-15 10:10:41 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually. 2022-07-15 10:10:41 [WARNING] Found possible input node names: [‘attention_mask’, ‘input_ids’], output node names: [‘global_average_pooling1d’]. WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually. 2022-07-15 10:10:44 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually. WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually. 2022-07-15 10:10:46 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually. WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually. 2022-07-15 10:10:48 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually. WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually. 2022-07-15 10:10:51 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually. 2022-07-15 10:10:51 [WARNING] Found possible input node names: [‘attention_mask’, ‘input_ids’], output node names: [‘global_average_pooling1d’]. WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually. 2022-07-15 10:10:54 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually. 2022-07-15 10:10:54.913870: I tensorflow/core/grappler/devices.cc:75] Number of eligible GPUs (core count >= 8, compute capability >= 2022-07-15 10:10:54.914104: I tensorflow/core/grappler/clusters/single_machine.cc:358] Starting new session 2022-07-15 10:10:56.035642: I tensorflow/core/grappler/devices.cc:75] Number of eligible GPUs (core count >= 8, compute capability >= 2022-07-15 10:10:56.035843: I tensorflow/core/grappler/clusters/single_machine.cc:358] Starting new session 2022-07-15 10:11:04 [INFO] Pass Quantization elapsed time: 5884.0 ms 2022-07-15 10:11:07 [INFO] Start sampling on calibration dataset. 2022-07-15 10:11:07.605083: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:354] MLIR V1 optimization pass is not enabled Traceback (most recent call last): File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py”, line 1377, in _do_call return fn(*args) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py”, line 1361, in _run_fn target_list, run_metadata) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py”, line 1455, in _call_tf_sessionrun run_metadata) tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[0,0] = is not in [0, ) [[{{node text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup}}]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File “/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py”, line 573, in quantize self._inference(self._sampling_model) File “/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py”, line 181, in _inference _ = model.sess.run(output_tensor, feed_dict) if model.iter_op==[]
File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py”, line 968, in run run_metadata_ptr) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py”, line 1191, in _run feed_dict_tensor, options, run_metadata) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py”, line 1371, in _do_run run_metadata) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py”, line 1396, in _do_call raise type(e)(node_def, op, message) # pylint: disable=no-value-for-parameter tensorflow.python.framework.errors_impl.InvalidArgumentError: Graph execution error:

Detected at node ‘text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup’ defined at (most recent call last): File “/opt/conda/lib/python3.7/runpy.py”, line 193, in _run_module_as_main “main”, mod_spec) File “/opt/conda/lib/python3.7/runpy.py”, line 85, in _run_code exec(code, run_globals) File “/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py”, line 17, in <module> app.launch_new_instance() File “/opt/conda/lib/python3.7/site-packages/traitlets/config/application.py”, line 976, in launch_instance app.start() File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelapp.py”, line 712, in start self.io_loop.start() File “/opt/conda/lib/python3.7/site-packages/tornado/platform/asyncio.py”, line 199, in start self.asyncio_loop.run_forever() File “/opt/conda/lib/python3.7/asyncio/base_events.py”, line 541, in run_forever self._run_once() File “/opt/conda/lib/python3.7/asyncio/base_events.py”, line 1786, in _run_once handle._run() File “/opt/conda/lib/python3.7/asyncio/events.py”, line 88, in _run self._context.run(self._callback, *self._args) File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 510, in dispatch_queue await self.process_one() File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 499, in process_one await dispatch(*args) File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 406, in dispatch_shell await result File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 730, in execute_request reply_content = await reply_content File “/opt/conda/lib/python3.7/site-packages/ipykernel/ipkernel.py”, line 387, in do_execute cell_id=cell_id, File “/opt/conda/lib/python3.7/site-packages/ipykernel/zmqshell.py”, line 528, in run_cell return super().run_cell(*args, **kwargs) File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 2975, in run_cell raw_cell, store_history, silent, shell_futures, cell_id File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3029, in _run_cell return runner(coro) File “/opt/conda/lib/python3.7/site-packages/IPython/core/async_helpers.py”, line 78, in pseudo_sync_runner coro.send(None) File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3257, in run_cell_async interactivity=interactivity, compiler=compiler, result=result) File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3472, in run_ast_nodes if (await self.run_code(code, result, async=asy)): File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3552, in run_code exec(code_obj, self.user_global_ns, self.user_ns) File “/tmp/ipykernel_28346/3554570389.py”, line 5, in <module> quantized_model = quantizer.fit() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py”, line 212, in call return super(Quantization, self).call() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/component.py”, line 215, in call results = self.execute() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py”, line 151, in execute self.strategy.traverse() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/strategy/strategy.py”, line 393, in traverse tune_cfg, self.model, self.calib_dataloader, self.q_func) File “/opt/conda/lib/python3.7/site-packages/neural_compressor/utils/utility.py”, line 242, in fi res = func(*args, **kwargs) File “/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tensorflow.py”, line 536, in quantize data_loader=data_loader).convert() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py”, line 268, in convert model = self.quantize() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py”, line 569, in quantize self._sampling_model.graph_def = sampling_graph_def File “/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py”, line 715, in graph_def self._output_tensor_names) File “/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py”, line 298, in graph_def_session tf.import_graph_def(model, name=‘’) Node: ‘text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup’ indices[0,0] = is not in [0, ) [[{{node text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup}}]]

Original stack trace for ‘text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup’: File “/opt/conda/lib/python3.7/runpy.py”, line 193, in _run_module_as_main “main”, mod_spec) File “/opt/conda/lib/python3.7/runpy.py”, line 85, in _run_code exec(code, run_globals) File “/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py”, line 17, in <module> app.launch_new_instance() File “/opt/conda/lib/python3.7/site-packages/traitlets/config/application.py”, line 976, in launch_instance app.start() File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelapp.py”, line 712, in start self.io_loop.start() File “/opt/conda/lib/python3.7/site-packages/tornado/platform/asyncio.py”, line 199, in start self.asyncio_loop.run_forever() File “/opt/conda/lib/python3.7/asyncio/base_events.py”, line 541, in run_forever self._run_once() File “/opt/conda/lib/python3.7/asyncio/base_events.py”, line 1786, in _run_once handle._run() File “/opt/conda/lib/python3.7/asyncio/events.py”, line 88, in _run self._context.run(self._callback, *self._args) File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 510, in dispatch_queue await self.process_one() File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 499, in process_one await dispatch(*args) File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 406, in dispatch_shell await result File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 730, in execute_request reply_content = await reply_content File “/opt/conda/lib/python3.7/site-packages/ipykernel/ipkernel.py”, line 387, in do_execute cell_id=cell_id, File “/opt/conda/lib/python3.7/site-packages/ipykernel/zmqshell.py”, line 528, in run_cell return super().run_cell(*args, **kwargs) File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 2975, in run_cell raw_cell, store_history, silent, shell_futures, cell_id File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3029, in _run_cell return runner(coro) File “/opt/conda/lib/python3.7/site-packages/IPython/core/async_helpers.py”, line 78, in pseudo_sync_runner coro.send(None) File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3257, in run_cell_async interactivity=interactivity, compiler=compiler, result=result) File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3472, in run_ast_nodes if (await self.run_code(code, result, async=asy)): File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3552, in run_code exec(code_obj, self.user_global_ns, self.user_ns) File “/tmp/ipykernel_28346/3554570389.py”, line 5, in <module> quantized_model = quantizer.fit() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py”, line 212, in call return super(Quantization, self).call() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/component.py”, line 215, in call results = self.execute() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py”, line 151, in execute self.strategy.traverse() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/strategy/strategy.py”, line 393, in traverse tune_cfg, self.model, self.calib_dataloader, self.q_func) File “/opt/conda/lib/python3.7/site-packages/neural_compressor/utils/utility.py”, line 242, in fi res = func(*args, **kwargs) File “/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tensorflow.py”, line 536, in quantize data_loader=data_loader).convert() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py”, line 268, in convert model = self.quantize() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py”, line 569, in quantize self._sampling_model.graph_def = sampling_graph_def File “/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py”, line 715, in graph_def self._output_tensor_names) File “/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py”, line 298, in graph_def_session tf.import_graph_def(model, name=‘’) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py”, line 561, in new_func return func(*args, **kwargs) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py”, line 408, in import_graph_def producer_op_list=producer_op_list) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py”, line 516, in _import_graph_def_internal _ProcessNewOps(graph) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py”, line 247, in _ProcessNewOps for new_op in graph._add_new_tf_operations(compute_devices=False): # pylint: disable=protected-access File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py”, line 3906, in _add_new_tf_operations for c_op in c_api_util.new_tf_operations(self) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py”, line 3906, in <listcomp> for c_op in c_api_util.new_tf_operations(self) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py”, line 3787, in _create_op_from_tf_operation ret = Operation(c_op, self) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py”, line 2133, in init self._traceback = tf_stack.extract_stack_for_node(self._c_op)

2022-07-15 10:11:08 [ERROR] Fail to quantize graph due to Graph execution error:

Detected at node ‘text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup’ defined at (most recent call last): File “/opt/conda/lib/python3.7/runpy.py”, line 193, in _run_module_as_main “main”, mod_spec) File “/opt/conda/lib/python3.7/runpy.py”, line 85, in _run_code exec(code, run_globals) File “/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py”, line 17, in <module> app.launch_new_instance() File “/opt/conda/lib/python3.7/site-packages/traitlets/config/application.py”, line 976, in launch_instance app.start() File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelapp.py”, line 712, in start self.io_loop.start() File “/opt/conda/lib/python3.7/site-packages/tornado/platform/asyncio.py”, line 199, in start self.asyncio_loop.run_forever() File “/opt/conda/lib/python3.7/asyncio/base_events.py”, line 541, in run_forever self._run_once() File “/opt/conda/lib/python3.7/asyncio/base_events.py”, line 1786, in _run_once handle._run() File “/opt/conda/lib/python3.7/asyncio/events.py”, line 88, in _run self._context.run(self._callback, *self._args) File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 510, in dispatch_queue await self.process_one() File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 499, in process_one await dispatch(*args) File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 406, in dispatch_shell await result File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 730, in execute_request reply_content = await reply_content File “/opt/conda/lib/python3.7/site-packages/ipykernel/ipkernel.py”, line 387, in do_execute cell_id=cell_id, File “/opt/conda/lib/python3.7/site-packages/ipykernel/zmqshell.py”, line 528, in run_cell return super().run_cell(*args, **kwargs) File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 2975, in run_cell raw_cell, store_history, silent, shell_futures, cell_id File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3029, in _run_cell return runner(coro) File “/opt/conda/lib/python3.7/site-packages/IPython/core/async_helpers.py”, line 78, in pseudo_sync_runner coro.send(None) File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3257, in run_cell_async interactivity=interactivity, compiler=compiler, result=result) File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3472, in run_ast_nodes if (await self.run_code(code, result, async=asy)): File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3552, in run_code exec(code_obj, self.user_global_ns, self.user_ns) File “/tmp/ipykernel_28346/3554570389.py”, line 5, in <module> quantized_model = quantizer.fit() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py”, line 212, in call return super(Quantization, self).call() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/component.py”, line 215, in call results = self.execute() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py”, line 151, in execute self.strategy.traverse() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/strategy/strategy.py”, line 393, in traverse tune_cfg, self.model, self.calib_dataloader, self.q_func) File “/opt/conda/lib/python3.7/site-packages/neural_compressor/utils/utility.py”, line 242, in fi res = func(*args, **kwargs) File “/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tensorflow.py”, line 536, in quantize data_loader=data_loader).convert() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py”, line 268, in convert model = self.quantize() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py”, line 569, in quantize self._sampling_model.graph_def = sampling_graph_def File “/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py”, line 715, in graph_def self._output_tensor_names) File “/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py”, line 298, in graph_def_session tf.import_graph_def(model, name=‘’) Node: ‘text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup’ indices[0,0] = is not in [0, ) [[{{node text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup}}]]

Original stack trace for ‘text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup’: File “/opt/conda/lib/python3.7/runpy.py”, line 193, in _run_module_as_main “main”, mod_spec) File “/opt/conda/lib/python3.7/runpy.py”, line 85, in _run_code exec(code, run_globals) File “/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py”, line 17, in <module> app.launch_new_instance() File “/opt/conda/lib/python3.7/site-packages/traitlets/config/application.py”, line 976, in launch_instance app.start() File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelapp.py”, line 712, in start self.io_loop.start() File “/opt/conda/lib/python3.7/site-packages/tornado/platform/asyncio.py”, line 199, in start self.asyncio_loop.run_forever() File “/opt/conda/lib/python3.7/asyncio/base_events.py”, line 541, in run_forever self._run_once() File “/opt/conda/lib/python3.7/asyncio/base_events.py”, line 1786, in _run_once handle._run() File “/opt/conda/lib/python3.7/asyncio/events.py”, line 88, in _run self._context.run(self._callback, *self._args) File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 510, in dispatch_queue await self.process_one() File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 499, in process_one await dispatch(*args) File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 406, in dispatch_shell await result File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 730, in execute_request reply_content = await reply_content File “/opt/conda/lib/python3.7/site-packages/ipykernel/ipkernel.py”, line 387, in do_execute cell_id=cell_id, File “/opt/conda/lib/python3.7/site-packages/ipykernel/zmqshell.py”, line 528, in run_cell return super().run_cell(*args, **kwargs) File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 2975, in run_cell raw_cell, store_history, silent, shell_futures, cell_id File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3029, in _run_cell return runner(coro) File “/opt/conda/lib/python3.7/site-packages/IPython/core/async_helpers.py”, line 78, in pseudo_sync_runner coro.send(None) File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3257, in run_cell_async interactivity=interactivity, compiler=compiler, result=result) File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3472, in run_ast_nodes if (await self.run_code(code, result, async=asy)): File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3552, in run_code exec(code_obj, self.user_global_ns, self.user_ns) File “/tmp/ipykernel_28346/3554570389.py”, line 5, in <module> quantized_model = quantizer.fit() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py”, line 212, in call return super(Quantization, self).call() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/component.py”, line 215, in call results = self.execute() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py”, line 151, in execute self.strategy.traverse() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/strategy/strategy.py”, line 393, in traverse tune_cfg, self.model, self.calib_dataloader, self.q_func) File “/opt/conda/lib/python3.7/site-packages/neural_compressor/utils/utility.py”, line 242, in fi res = func(*args, **kwargs) File “/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tensorflow.py”, line 536, in quantize data_loader=data_loader).convert() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py”, line 268, in convert model = self.quantize() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py”, line 569, in quantize self._sampling_model.graph_def = sampling_graph_def File “/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py”, line 715, in graph_def self._output_tensor_names) File “/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py”, line 298, in graph_def_session tf.import_graph_def(model, name=‘’) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py”, line 561, in new_func return func(*args, **kwargs) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py”, line 408, in import_graph_def producer_op_list=producer_op_list) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py”, line 516, in _import_graph_def_internal _ProcessNewOps(graph) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py”, line 247, in _ProcessNewOps for new_op in graph._add_new_tf_operations(compute_devices=False): # pylint: disable=protected-access File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py”, line 3906, in _add_new_tf_operations for c_op in c_api_util.new_tf_operations(self) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py”, line 3906, in <listcomp> for c_op in c_api_util.new_tf_operations(self) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py”, line 3787, in _create_op_from_tf_operation ret = Operation(c_op, self) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py”, line 2133, in init self._traceback = tf_stack.extract_stack_for_node(self._c_op) . 2022-07-15 10:11:08 [WARNING] Fail to forward with batch size=1, set to 1 now. WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually. 2022-07-15 10:11:10 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually. WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually. 2022-07-15 10:11:13 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually. WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually. 2022-07-15 10:11:15 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually. WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually. 2022-07-15 10:11:18 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually. 2022-07-15 10:11:18 [WARNING] Found possible input node names: [‘attention_mask’, ‘input_ids’], output node names: [‘global_average_pooling1d’]. WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually. 2022-07-15 10:11:21 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually. WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually. 2022-07-15 10:11:23 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually. WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually. 2022-07-15 10:11:25 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually. WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually. 2022-07-15 10:11:27 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually. 2022-07-15 10:11:28 [WARNING] Found possible input node names: [‘attention_mask’, ‘input_ids’], output node names: [‘global_average_pooling1d’]. WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually. 2022-07-15 10:11:31 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually. 2022-07-15 10:11:31.485104: I tensorflow/core/grappler/devices.cc:75] Number of eligible GPUs (core count >= 8, compute capability >= 2022-07-15 10:11:31.485309: I tensorflow/core/grappler/clusters/single_machine.cc:358] Starting new session 2022-07-15 10:11:32.564946: I tensorflow/core/grappler/devices.cc:75] Number of eligible GPUs (core count >= 8, compute capability >= 2022-07-15 10:11:32.565120: I tensorflow/core/grappler/clusters/single_machine.cc:358] Starting new session 2022-07-15 10:11:40 [INFO] Pass Quantization elapsed time: 5611.09 ms 2022-07-15 10:11:43 [INFO] Start sampling on calibration dataset. Traceback (most recent call last): File “/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tensorflow.py”, line 536, in quantize data_loader=data_loader).convert() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py”, line 275, in convert post_cse_graph_def = PostCseOptimizer(model.graph_def).do_transformation() AttributeError: ‘NoneType’ object has no attribute ‘graph_def’

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py”, line 1377, in _do_call return fn(*args) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py”, line 1361, in _run_fn target_list, run_metadata) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py”, line 1455, in _call_tf_sessionrun run_metadata) tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[0,0] = is not in [0, ) [[{{node text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup}}]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File “/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py”, line 573, in quantize self._inference(self._sampling_model) File “/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py”, line 181, in _inference _ = model.sess.run(output_tensor, feed_dict) if model.iter_op==[]
File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py”, line 968, in run run_metadata_ptr) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py”, line 1191, in _run feed_dict_tensor, options, run_metadata) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py”, line 1371, in _do_run run_metadata) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py”, line 1396, in _do_call raise type(e)(node_def, op, message) # pylint: disable=no-value-for-parameter tensorflow.python.framework.errors_impl.InvalidArgumentError: Graph execution error:

Detected at node ‘text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup’ defined at (most recent call last): File “/opt/conda/lib/python3.7/runpy.py”, line 193, in _run_module_as_main “main”, mod_spec) File “/opt/conda/lib/python3.7/runpy.py”, line 85, in _run_code exec(code, run_globals) File “/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py”, line 17, in <module> app.launch_new_instance() File “/opt/conda/lib/python3.7/site-packages/traitlets/config/application.py”, line 976, in launch_instance app.start() File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelapp.py”, line 712, in start self.io_loop.start() File “/opt/conda/lib/python3.7/site-packages/tornado/platform/asyncio.py”, line 199, in start self.asyncio_loop.run_forever() File “/opt/conda/lib/python3.7/asyncio/base_events.py”, line 541, in run_forever self._run_once() File “/opt/conda/lib/python3.7/asyncio/base_events.py”, line 1786, in _run_once handle._run() File “/opt/conda/lib/python3.7/asyncio/events.py”, line 88, in _run self._context.run(self._callback, *self._args) File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 510, in dispatch_queue await self.process_one() File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 499, in process_one await dispatch(*args) File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 406, in dispatch_shell await result File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 730, in execute_request reply_content = await reply_content File “/opt/conda/lib/python3.7/site-packages/ipykernel/ipkernel.py”, line 387, in do_execute cell_id=cell_id, File “/opt/conda/lib/python3.7/site-packages/ipykernel/zmqshell.py”, line 528, in run_cell return super().run_cell(*args, **kwargs) File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 2975, in run_cell raw_cell, store_history, silent, shell_futures, cell_id File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3029, in _run_cell return runner(coro) File “/opt/conda/lib/python3.7/site-packages/IPython/core/async_helpers.py”, line 78, in pseudo_sync_runner coro.send(None) File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3257, in run_cell_async interactivity=interactivity, compiler=compiler, result=result) File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3472, in run_ast_nodes if (await self.run_code(code, result, async=asy)): File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3552, in run_code exec(code_obj, self.user_global_ns, self.user_ns) File “/tmp/ipykernel_28346/3554570389.py”, line 5, in <module> quantized_model = quantizer.fit() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py”, line 212, in call return super(Quantization, self).call() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/component.py”, line 215, in call results = self.execute() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py”, line 151, in execute self.strategy.traverse() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/strategy/strategy.py”, line 393, in traverse tune_cfg, self.model, self.calib_dataloader, self.q_func) File “/opt/conda/lib/python3.7/site-packages/neural_compressor/utils/utility.py”, line 242, in fi res = func(*args, **kwargs) File “/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tensorflow.py”, line 551, in quantize data_loader=data_loader).convert() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py”, line 268, in convert model = self.quantize() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py”, line 569, in quantize self._sampling_model.graph_def = sampling_graph_def File “/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py”, line 715, in graph_def self._output_tensor_names) File “/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py”, line 298, in graph_def_session tf.import_graph_def(model, name=‘’) Node: ‘text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup’ indices[0,0] = is not in [0, ) [[{{node text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup}}]]

Original stack trace for ‘text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup’: File “/opt/conda/lib/python3.7/runpy.py”, line 193, in _run_module_as_main “main”, mod_spec) File “/opt/conda/lib/python3.7/runpy.py”, line 85, in _run_code exec(code, run_globals) File “/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py”, line 17, in <module> app.launch_new_instance() File “/opt/conda/lib/python3.7/site-packages/traitlets/config/application.py”, line 976, in launch_instance app.start() File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelapp.py”, line 712, in start self.io_loop.start() File “/opt/conda/lib/python3.7/site-packages/tornado/platform/asyncio.py”, line 199, in start self.asyncio_loop.run_forever() File “/opt/conda/lib/python3.7/asyncio/base_events.py”, line 541, in run_forever self._run_once() File “/opt/conda/lib/python3.7/asyncio/base_events.py”, line 1786, in _run_once handle._run() File “/opt/conda/lib/python3.7/asyncio/events.py”, line 88, in _run self._context.run(self._callback, *self._args) File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 510, in dispatch_queue await self.process_one() File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 499, in process_one await dispatch(*args) File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 406, in dispatch_shell await result File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 730, in execute_request reply_content = await reply_content File “/opt/conda/lib/python3.7/site-packages/ipykernel/ipkernel.py”, line 387, in do_execute cell_id=cell_id, File “/opt/conda/lib/python3.7/site-packages/ipykernel/zmqshell.py”, line 528, in run_cell return super().run_cell(*args, **kwargs) File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 2975, in run_cell raw_cell, store_history, silent, shell_futures, cell_id File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3029, in _run_cell return runner(coro) File “/opt/conda/lib/python3.7/site-packages/IPython/core/async_helpers.py”, line 78, in pseudo_sync_runner coro.send(None) File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3257, in run_cell_async interactivity=interactivity, compiler=compiler, result=result) File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3472, in run_ast_nodes if (await self.run_code(code, result, async=asy)): File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3552, in run_code exec(code_obj, self.user_global_ns, self.user_ns) File “/tmp/ipykernel_28346/3554570389.py”, line 5, in <module> quantized_model = quantizer.fit() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py”, line 212, in call return super(Quantization, self).call() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/component.py”, line 215, in call results = self.execute() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py”, line 151, in execute self.strategy.traverse() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/strategy/strategy.py”, line 393, in traverse tune_cfg, self.model, self.calib_dataloader, self.q_func) File “/opt/conda/lib/python3.7/site-packages/neural_compressor/utils/utility.py”, line 242, in fi res = func(*args, **kwargs) File “/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tensorflow.py”, line 551, in quantize data_loader=data_loader).convert() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py”, line 268, in convert model = self.quantize() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py”, line 569, in quantize self._sampling_model.graph_def = sampling_graph_def File “/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py”, line 715, in graph_def self._output_tensor_names) File “/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py”, line 298, in graph_def_session tf.import_graph_def(model, name=‘’) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py”, line 561, in new_func return func(*args, **kwargs) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py”, line 408, in import_graph_def producer_op_list=producer_op_list) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py”, line 516, in _import_graph_def_internal _ProcessNewOps(graph) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py”, line 247, in _ProcessNewOps for new_op in graph._add_new_tf_operations(compute_devices=False): # pylint: disable=protected-access File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py”, line 3906, in _add_new_tf_operations for c_op in c_api_util.new_tf_operations(self) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py”, line 3906, in <listcomp> for c_op in c_api_util.new_tf_operations(self) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py”, line 3787, in _create_op_from_tf_operation ret = Operation(c_op, self) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py”, line 2133, in init self._traceback = tf_stack.extract_stack_for_node(self._c_op)

2022-07-15 10:11:43 [ERROR] Fail to quantize graph due to Graph execution error:

Detected at node ‘text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup’ defined at (most recent call last): File “/opt/conda/lib/python3.7/runpy.py”, line 193, in _run_module_as_main “main”, mod_spec) File “/opt/conda/lib/python3.7/runpy.py”, line 85, in _run_code exec(code, run_globals) File “/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py”, line 17, in <module> app.launch_new_instance() File “/opt/conda/lib/python3.7/site-packages/traitlets/config/application.py”, line 976, in launch_instance app.start() File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelapp.py”, line 712, in start self.io_loop.start() File “/opt/conda/lib/python3.7/site-packages/tornado/platform/asyncio.py”, line 199, in start self.asyncio_loop.run_forever() File “/opt/conda/lib/python3.7/asyncio/base_events.py”, line 541, in run_forever self._run_once() File “/opt/conda/lib/python3.7/asyncio/base_events.py”, line 1786, in _run_once handle._run() File “/opt/conda/lib/python3.7/asyncio/events.py”, line 88, in _run self._context.run(self._callback, *self._args) File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 510, in dispatch_queue await self.process_one() File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 499, in process_one await dispatch(*args) File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 406, in dispatch_shell await result File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 730, in execute_request reply_content = await reply_content File “/opt/conda/lib/python3.7/site-packages/ipykernel/ipkernel.py”, line 387, in do_execute cell_id=cell_id, File “/opt/conda/lib/python3.7/site-packages/ipykernel/zmqshell.py”, line 528, in run_cell return super().run_cell(*args, **kwargs) File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 2975, in run_cell raw_cell, store_history, silent, shell_futures, cell_id File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3029, in _run_cell return runner(coro) File “/opt/conda/lib/python3.7/site-packages/IPython/core/async_helpers.py”, line 78, in pseudo_sync_runner coro.send(None) File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3257, in run_cell_async interactivity=interactivity, compiler=compiler, result=result) File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3472, in run_ast_nodes if (await self.run_code(code, result, async=asy)): File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3552, in run_code exec(code_obj, self.user_global_ns, self.user_ns) File “/tmp/ipykernel_28346/3554570389.py”, line 5, in <module> quantized_model = quantizer.fit() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py”, line 212, in call return super(Quantization, self).call() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/component.py”, line 215, in call results = self.execute() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py”, line 151, in execute self.strategy.traverse() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/strategy/strategy.py”, line 393, in traverse tune_cfg, self.model, self.calib_dataloader, self.q_func) File “/opt/conda/lib/python3.7/site-packages/neural_compressor/utils/utility.py”, line 242, in fi res = func(*args, **kwargs) File “/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tensorflow.py”, line 551, in quantize data_loader=data_loader).convert() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py”, line 268, in convert model = self.quantize() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py”, line 569, in quantize self._sampling_model.graph_def = sampling_graph_def File “/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py”, line 715, in graph_def self._output_tensor_names) File “/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py”, line 298, in graph_def_session tf.import_graph_def(model, name=‘’) Node: ‘text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup’ indices[0,0] = is not in [0, ) [[{{node text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup}}]]

Original stack trace for ‘text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup’: File “/opt/conda/lib/python3.7/runpy.py”, line 193, in _run_module_as_main “main”, mod_spec) File “/opt/conda/lib/python3.7/runpy.py”, line 85, in _run_code exec(code, run_globals) File “/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py”, line 17, in <module> app.launch_new_instance() File “/opt/conda/lib/python3.7/site-packages/traitlets/config/application.py”, line 976, in launch_instance app.start() File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelapp.py”, line 712, in start self.io_loop.start() File “/opt/conda/lib/python3.7/site-packages/tornado/platform/asyncio.py”, line 199, in start self.asyncio_loop.run_forever() File “/opt/conda/lib/python3.7/asyncio/base_events.py”, line 541, in run_forever self._run_once() File “/opt/conda/lib/python3.7/asyncio/base_events.py”, line 1786, in _run_once handle._run() File “/opt/conda/lib/python3.7/asyncio/events.py”, line 88, in _run self._context.run(self._callback, *self._args) File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 510, in dispatch_queue await self.process_one() File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 499, in process_one await dispatch(*args) File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 406, in dispatch_shell await result File “/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 730, in execute_request reply_content = await reply_content File “/opt/conda/lib/python3.7/site-packages/ipykernel/ipkernel.py”, line 387, in do_execute cell_id=cell_id, File “/opt/conda/lib/python3.7/site-packages/ipykernel/zmqshell.py”, line 528, in run_cell return super().run_cell(*args, **kwargs) File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 2975, in run_cell raw_cell, store_history, silent, shell_futures, cell_id File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3029, in _run_cell return runner(coro) File “/opt/conda/lib/python3.7/site-packages/IPython/core/async_helpers.py”, line 78, in pseudo_sync_runner coro.send(None) File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3257, in run_cell_async interactivity=interactivity, compiler=compiler, result=result) File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3472, in run_ast_nodes if (await self.run_code(code, result, async=asy)): File “/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3552, in run_code exec(code_obj, self.user_global_ns, self.user_ns) File “/tmp/ipykernel_28346/3554570389.py”, line 5, in <module> quantized_model = quantizer.fit() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py”, line 212, in call return super(Quantization, self).call() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/component.py”, line 215, in call results = self.execute() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py”, line 151, in execute self.strategy.traverse() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/strategy/strategy.py”, line 393, in traverse tune_cfg, self.model, self.calib_dataloader, self.q_func) File “/opt/conda/lib/python3.7/site-packages/neural_compressor/utils/utility.py”, line 242, in fi res = func(*args, **kwargs) File “/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tensorflow.py”, line 551, in quantize data_loader=data_loader).convert() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py”, line 268, in convert model = self.quantize() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py”, line 569, in quantize self._sampling_model.graph_def = sampling_graph_def File “/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py”, line 715, in graph_def self._output_tensor_names) File “/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py”, line 298, in graph_def_session tf.import_graph_def(model, name=‘’) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py”, line 561, in new_func return func(*args, **kwargs) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py”, line 408, in import_graph_def producer_op_list=producer_op_list) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py”, line 516, in _import_graph_def_internal _ProcessNewOps(graph) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py”, line 247, in _ProcessNewOps for new_op in graph._add_new_tf_operations(compute_devices=False): # pylint: disable=protected-access File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py”, line 3906, in _add_new_tf_operations for c_op in c_api_util.new_tf_operations(self) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py”, line 3906, in <listcomp> for c_op in c_api_util.new_tf_operations(self) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py”, line 3787, in _create_op_from_tf_operation ret = Operation(c_op, self) File “/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py”, line 2133, in init self._traceback = tf_stack.extract_stack_for_node(self._c_op) . 2022-07-15 10:11:43 [ERROR] Unexpected exception AttributeError(“‘NoneType’ object has no attribute ‘graph_def’”) happened during tuning. Traceback (most recent call last): File “/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tensorflow.py”, line 536, in quantize data_loader=data_loader).convert() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py”, line 275, in convert post_cse_graph_def = PostCseOptimizer(model.graph_def).do_transformation() AttributeError: ‘NoneType’ object has no attribute ‘graph_def’

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File “/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py”, line 151, in execute self.strategy.traverse() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/strategy/strategy.py”, line 393, in traverse tune_cfg, self.model, self.calib_dataloader, self.q_func) File “/opt/conda/lib/python3.7/site-packages/neural_compressor/utils/utility.py”, line 242, in fi res = func(*args, **kwargs) File “/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tensorflow.py”, line 551, in quantize data_loader=data_loader).convert() File “/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py”, line 275, in convert post_cse_graph_def = PostCseOptimizer(model.graph_def).do_transformation() AttributeError: ‘NoneType’ object has no attribute ‘graph_def’ 2022-07-15 10:11:43 [ERROR] Specified timeout or max trials is reached! Not found any quantized model which meet accuracy goal. Exit.

AttributeError Traceback (most recent call last) /tmp/ipykernel_28346/3554570389.py in <module> 5 quantized_model = quantizer.fit() 6 output_path = ‘…/models/gev8_image_encoder_quant/’ ----> 7 quantized_model.save(output_path)

AttributeError: ‘NoneType’ object has no attribute ‘save’

Any suggestion?