Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Transformer-XL: Convert lm1b model to PyTorch

See original GitHub issue

Hi,

I wanted to convert the TensorFlow checkpoint for the lm1b model to PyTorch with the convert_transfo_xl_checkpoint_to_pytorch.py script.

I downloaded the checkpoint with the download.sh script.

Then I called the convert script with:

$ python3 convert_transfo_xl_checkpoint_to_pytorch.py --pytorch_dump_folder_path converted --tf_checkpoint_path
/mnt/transformer-xl/tf/sota/pretrained_xl/tf_lm1b/model/checkpoint

Then the following error message is returned:

2019-02-19 22:46:54.693060: W tensorflow/core/util/tensor_slice_reader.cc:95] Could not open /mnt/transformer-xl/tf/sota/pretrained_xl/tf_lm1b/model/checkpoint: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?
Traceback (most recent call last):
  File "convert_transfo_xl_checkpoint_to_pytorch.py", line 116, in <module>
    args.transfo_xl_dataset_file)
  File "convert_transfo_xl_checkpoint_to_pytorch.py", line 81, in convert_transfo_xl_checkpoint_to_pytorch
    model = load_tf_weights_in_transfo_xl(model, config, tf_path)
  File "/usr/local/lib/python3.6/dist-packages/pytorch_pretrained_bert/modeling_transfo_xl.py", line 141, in load_tf_weights_in_transfo_xl
    init_vars = tf.train.list_variables(tf_path)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/training/checkpoint_utils.py", line 95, in list_variables
    reader = load_checkpoint(ckpt_dir_or_file)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/training/checkpoint_utils.py", line 64, in load_checkpoint
    return pywrap_tensorflow.NewCheckpointReader(filename)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 382, in NewCheckpointReader
    return CheckpointReader(compat.as_bytes(filepattern), status)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/errors_impl.py", line 548, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.DataLossError: Unable to open table file /mnt/transformer-xl/tf/sota/pretrained_xl/tf_lm1b/model/checkpoint: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?

I’m using the 0.6.1 version of pytorch-pretrained-BERT and the latest tf-nightly-gpu package that ships TensorFlow 1.13dev.

Issue Analytics

State:
Created 5 years ago
Comments:12 (3 by maintainers)

Top GitHub Comments

1reaction

stefan-itcommented, Dec 9, 2019

Hi @irugina, unfortunately, I wasn’t able to convert the model 😞

1reaction

stefan-itcommented, Mar 6, 2019

Sadly, I couldn’t managed to convert it (I tried several options)

Top Results From Across the Web

Transformer-XL: Convert lm1b model to PyTorch #298 - GitHub

Hi, I wanted to convert the TensorFlow checkpoint for the lm1b model to PyTorch with the convert_transfo_xl_checkpoint_to_pytorch.py script.

Transformer XL - Hugging Face

The Transformer-XL model was proposed in Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context by Zihang Dai, Zhilin Yang, Yiming Yang, ...

Transformer-XL for PyTorch - NVIDIA NGC

Transformer -XL is a transformer-based language model with a segment-level recurrence and a novel relative positional encoding.

Evaluating Metrics of Language Model Performance for ...

els on LM1B in order to reproduce and facilitate ... official PyTorch website (Paszke et al., 2017).3 ... The Transformer-XL model introduces architec-....

Model Zoo - Deep learning code and pretrained models for ...

Find models that you need, for educational purposes, transfer learning, ... A PyTorch implementation of BigGAN with pretrained weights and conversion ...