Last release trigger ModuleNotFoundError: No module named '_lzma'

Since the last release, our tensorflow/datasets tests are failing with E ModuleNotFoundError: No module named '_lzma'. Triggered by:

/home/kbuilder/virtualenv/tfds_env/lib/python3.6/site-packages/fastavro/_read_py.py:12: in <module>
    import lzma

Is it expected ? Is there a solution ?

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tensorflow_datasets/testing/dataset_builder_testing.py:305: in test_download_and_prepare_as_dataset
    self._download_and_prepare_as_dataset(builder)
tensorflow_datasets/testing/dataset_builder_testing.py:365: in _download_and_prepare_as_dataset
    import apache_beam as beam   # pylint: disable=import-outside-toplevel,g-import-not-at-top
/home/kbuilder/virtualenv/tfds_env/lib/python3.6/site-packages/apache_beam/__init__.py:95: in <module>
    from apache_beam import coders
/home/kbuilder/virtualenv/tfds_env/lib/python3.6/site-packages/apache_beam/coders/__init__.py:19: in <module>
    from apache_beam.coders.coders import *
/home/kbuilder/virtualenv/tfds_env/lib/python3.6/site-packages/apache_beam/coders/coders.py:46: in <module>
    from apache_beam.coders import coder_impl
apache_beam/coders/coder_impl.py:53: in init apache_beam.coders.coder_impl
    ???
/home/kbuilder/virtualenv/tfds_env/lib/python3.6/site-packages/fastavro/__init__.py:47: in <module>
    import fastavro.read
/home/kbuilder/virtualenv/tfds_env/lib/python3.6/site-packages/fastavro/read.py:4: in <module>
    from . import _read_py as _read  # type: ignore
/home/kbuilder/virtualenv/tfds_env/lib/python3.6/site-packages/fastavro/_read_py.py:12: in <module>
    import lzma
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    """

    __all__ = [
        "CHECK_NONE", "CHECK_CRC32", "CHECK_CRC64", "CHECK_SHA256",
        "CHECK_ID_MAX", "CHECK_UNKNOWN",
        "FILTER_LZMA1", "FILTER_LZMA2", "FILTER_DELTA", "FILTER_X86", "FILTER_IA64",
        "FILTER_ARM", "FILTER_ARMTHUMB", "FILTER_POWERPC", "FILTER_SPARC",
        "FORMAT_AUTO", "FORMAT_XZ", "FORMAT_ALONE", "FORMAT_RAW",
        "MF_HC3", "MF_HC4", "MF_BT2", "MF_BT3", "MF_BT4",
        "MODE_FAST", "MODE_NORMAL", "PRESET_DEFAULT", "PRESET_EXTREME",

        "LZMACompressor", "LZMADecompressor", "LZMAFile", "LZMAError",
        "open", "compress", "decompress", "is_check_supported",
    ]

    import builtins
    import io
    import os
>   from _lzma import *
E   ModuleNotFoundError: No module named '_lzma'

/home/kbuilder/.pyenv/versions/3.6.8/lib/python3.6/lzma.py:27: ModuleNotFoundError

Full logs: https://source.cloud.google.com/results/invocations/e89f8616-7250-42ee-bae5-0b4b01dd2f2e/targets/tensorflow_datasets%2Fgh_testing%2Fsubjobs%2Fpy3_tfnightly/log

[gshaikov]

Found solution, I am using 3.7.0, probably works with other versions:

pyenv local --unset
pyenv uninstall -f 3.7.0
sudo apt-get -y update
sudo apt-get -y install --no-install-recommends liblzma-dev
pyenv install -f 3.7.0

Better solution though is to update machine image you’re and install the lzma library before installing pipenv.

[Conchylicultor]

Closing this. Installing liblzma-dev works (with the caveat this has to be installed before Pyenv). If anything, this should be a Pyenv bug. Thank you for your help.