with op Tile must be a compile-time constant.

TPU tensorflow2.6.0

TFLongTransformer

model.fit( train_data, tf.gather(train_label, train_idx), validation_data=(val_data, tf.gather(train_label, val_idx)), epochs=10, batch_size=BATCH_SIZE, callbacks=[checkpoint_callback], )

---

InvalidArgumentError Traceback (most recent call last) <ipython-input-51-69b9068b95d6> in <module>() 21 epochs=10, 22 batch_size=BATCH_SIZE, —> 23 callbacks=[checkpoint_callback], 24 ) 25 # model.load_weights(‘/content/ckpt_’+str(fold)+‘_model.h5’)

13 frames /usr/local/lib/python3.7/dist-packages/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_batch_size, validation_freq, max_queue_size, workers, use_multiprocessing) 1187 logs = tmp_logs # No error, now safe to assign to logs. 1188 end_step = step + data_handler.step_increment -> 1189 callbacks.on_train_batch_end(end_step, logs) 1190 if self.stop_training: 1191 break

/usr/local/lib/python3.7/dist-packages/keras/callbacks.py in on_train_batch_end(self, batch, logs) 433 “”" 434 if self._should_call_train_batch_hooks: –> 435 self._call_batch_hook(ModeKeys.TRAIN, ‘end’, batch, logs=logs) 436 437 def on_test_batch_begin(self, batch, logs=None):

/usr/local/lib/python3.7/dist-packages/keras/callbacks.py in _call_batch_hook(self, mode, hook, batch, logs) 293 self._call_batch_begin_hook(mode, batch, logs) 294 elif hook == ‘end’: –> 295 self._call_batch_end_hook(mode, batch, logs) 296 else: 297 raise ValueError(‘Unrecognized hook: {}’.format(hook))

/usr/local/lib/python3.7/dist-packages/keras/callbacks.py in _call_batch_end_hook(self, mode, batch, logs) 313 self._batch_times.append(batch_time) 314 –> 315 self._call_batch_hook_helper(hook_name, batch, logs) 316 317 if len(self._batch_times) >= self._num_batches_for_timing_check:

/usr/local/lib/python3.7/dist-packages/keras/callbacks.py in _call_batch_hook_helper(self, hook_name, batch, logs) 351 for callback in self.callbacks: 352 hook = getattr(callback, hook_name) –> 353 hook(batch, logs) 354 355 if self._check_timing:

/usr/local/lib/python3.7/dist-packages/keras/callbacks.py in on_train_batch_end(self, batch, logs) 1026 1027 def on_train_batch_end(self, batch, logs=None): -> 1028 self._batch_update_progbar(batch, logs) 1029 1030 def on_test_batch_end(self, batch, logs=None):

/usr/local/lib/python3.7/dist-packages/keras/callbacks.py in _batch_update_progbar(self, batch, logs) 1098 if self.verbose == 1: 1099 # Only block async when verbose = 1. -> 1100 logs = tf_utils.sync_to_numpy_or_python_type(logs) 1101 self.progbar.update(self.seen, list(logs.items()), finalize=False) 1102

/usr/local/lib/python3.7/dist-packages/keras/utils/tf_utils.py in sync_to_numpy_or_python_type(tensors) 514 return t # Don’t turn ragged or sparse tensors to NumPy. 515 –> 516 return tf.nest.map_structure(_to_single_numpy_or_python_type, tensors) 517 518

/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/nest.py in map_structure(func, *structure, **kwargs) 867 868 return pack_sequence_as( –> 869 structure[0], [func(*x) for x in entries], 870 expand_composites=expand_composites) 871

/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/nest.py in <listcomp>(.0) 867 868 return pack_sequence_as( –> 869 structure[0], [func(*x) for x in entries], 870 expand_composites=expand_composites) 871

/usr/local/lib/python3.7/dist-packages/keras/utils/tf_utils.py in _to_single_numpy_or_python_type(t) 510 def _to_single_numpy_or_python_type(t): 511 if isinstance(t, tf.Tensor): –> 512 x = t.numpy() 513 return x.item() if np.ndim(x) == 0 else x 514 return t # Don’t turn ragged or sparse tensors to NumPy.

/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py in numpy(self) 1092 “”" 1093 # TODO(slebedev): Consider avoiding a copy for non-CPU or remote tensors. -> 1094 maybe_arr = self._numpy() # pylint: disable=protected-access 1095 return maybe_arr.copy() if isinstance(maybe_arr, np.ndarray) else maybe_arr 1096

/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py in _numpy(self) 1060 return self._numpy_internal() 1061 except core._NotOkStatusException as e: # pylint: disable=protected-access -> 1062 six.raise_from(core._status_to_exception(e.code, e.message), None) # pylint: disable=protected-access 1063 1064 @property

/usr/local/lib/python3.7/dist-packages/six.py in raise_from(value, from_value)

InvalidArgumentError: 9 root error(s) found. (0) Invalid argument: {{function_node __inference_train_function_367359}} Input 1 to node model/tf_longformer_model/longformer/encoder/layer_._0/attention/self/cond_1/Tile with op Tile must be a compile-time constant.

XLA compilation requires that operator arguments that represent shapes or dimensions be evaluated to concrete values at compile time. This error means that a shape or dimension argument could not be evaluated at compile time, usually because the value of the argument depends on a parameter to the computation, on a variable, or on a stateful operation such as a random number generator.

 [[{{node model/tf_longformer_model/longformer/encoder/layer_._0/attention/self/cond_1/Tile}}]]

 [[model/tf_longformer_model/longformer/encoder/layer_._0/attention/self/cond_1]]
 [[TPUReplicate/_compile/_13050048924464197292/_9]]
 [[TPUReplicate/_compile/_13050048924464197292/_9/_438]]

(1) Invalid argument: {{function_node __inference_train_function_367359}} Input 1 to node model/tf_longformer_model/longformer/encoder/layer_._0/attention/self/cond_1/Tile with op Tile must be a compile-time constant.

XLA compilation requires that operator arguments that represent shapes or dimensions be evaluated to concrete values at compile time. This error means that a shape or dimension argument could not be evaluated at compile time, usually because the value of the argument depends on a parameter to the computation, on a variable, or on a stateful operation such as a random number generator.

 [[{{node model/tf_longformer_model/longformer/encoder/layer_._0/attention/self/cond_1/Tile}}]]

 [[model/tf_longformer_model/longformer/encoder/layer_._0/attention/self/cond_1]]
 [[TPUReplicate/_compile/_13050048924464197292/_9]]
 [[tpu_compile_succeeded_assert/_1014831673988967065/_10/_397]]

(2) Invalid argument: {{function_node __inference_train_function_367359}} Input 1 to node model/tf_longformer_model/longformer/encoder/layer_._0/attention/self/cond_1/Tile with op Tile must be a compile-time constant.

XLA compilation requires that operator arguments that represent shapes or dimensions be evaluated to concrete values at compile time. This error means that a shape or dimension argument could not be evaluated at compile time, usually because the value of the argument depends on a parameter to the computation, on a variable, or on a stateful operation such as a random number generator.

 [[{{node model/tf_longformer_model/longformer/encoder/layer_._0/attention/self/cond_1/Tile}}]]

 [[model/tf_longformer_model/longformer/encoder/layer_._0/attention/self/cond_1]]
 [[TPUReplicate/_compile/_13050048924464197292/_9]]
 [[TPUReplicate/_compile/_13050048924464197292/_9/_416]]

(3) Invalid argument: {{function_node __inference_train_function_367359}} Input 1 to node model/tf_longformer_model/longformer/encoder/layer_._0/attention/self/cond_1/Tile with op Tile must be a compile-time constant.

XLA compilation requires that operator arguments that represent shapes or dimensions be evaluated to concre … [truncated]

[sarthak-314]

I have the same issue. I am guessing it’s because of random attention mechanism of Longformer

[gante]

Hi there! Possibly late for this issue, but an issue with the same error message (on a different model) was fixed here.

The issue was fixed by setting fixed shapes in the input.