Colab Demo Notebook Not Working

I launched the Colab notebook to try and demo this model.

There’s a section that starts with: Sometimes the next step errors for some reason, just run it again ¯\_(ツ)_/¯

That’s fine and all, except running it multiple times didn’t help. To try and resolve some errors, I tried to: !pip install optax transformers ray

That got me closer, but it errors out with:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-18-a22d9a83aa66> in <module>()
      7 import transformers
      8 
----> 9 from mesh_transformer.checkpoint import read_ckpt_lowmem
     10 from mesh_transformer.sampling import nucleaus_sample
     11 from mesh_transformer.transformer_shard import CausalTransformer

2 frames
/usr/lib/python3.7/typing.py in __new__(cls, *args, **kwds)
    308                 isinstance(args[1], tuple)):
    309             # Close enough.
--> 310             raise TypeError(f"Cannot subclass {cls!r}")
    311         return super().__new__(cls)
    312 

TypeError: Cannot subclass <class 'typing._SpecialForm'>

[JohnnyOpcode]

I beg to differ. It takes some effort to get the right mix of dependencies, but when it works it works quite well.

Here is what I use in my Colab Pro to bootstrap the model.

These are the packages that work and bypass requirements.txt (which is outdated/quirky on dependencies)

!pip install numpy~=1.21.0 !pip install typing-extensions~=3.7.4 !pip install tqdm>=4.45.0 !pip install wandb>=0.11.2 !pip install einops~=0.3.0 !pip install requests~=2.25.1 !pip install fabric~=2.6.0 !pip install optax==0.0.9 !pip install dm-haiku==0.0.5 !pip install git+https://github.com/EleutherAI/lm-evaluation-harness/ !pip install ray[default]==1.4.1 !pip install jaxlib~=0.1.68 !pip install jax~=0.2.12 !pip install Flask~=1.1.2 !pip install cloudpickle~=1.3.0 !pip install tensorflow-cpu~=2.7.0 !pip install google-cloud-storage~=1.36.2 !pip install transformers !pip install smart_open[gcs] !pip install func_timeout !pip install ftfy !pip install fastapi !pip install uvicorn !pip install lm_dataformat !pip install pathy

!pip install mesh-transformer-jax/ jax==0.2.12 tensorflow==2.7.0 chex==0.1.2 jaxlib==0.1.68

[Aspie96]

Well, it’s obvious that it needs attention. It’s all anyone needs.