question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Downloading pre-trained model during docker build

See original GitHub issue

Whenever we call SentenceTransformer(model_name), it downloads the pre-trained model from the server locally. But this happens during the runtime. I want to run this in a docker container and I want to know if there is any command which I can add inside the Dockerfile, so that the pre-trained model gets downloaded during the docker build itself ?

Issue Analytics

  • State:open
  • Created 3 years ago
  • Comments:15 (6 by maintainers)

github_iconTop GitHub Comments

11reactions
chanindcommented, Sep 20, 2021

Adding the following into my Dockerfile worked for downloading / setting up the model. It just causes the initial download to happen during the docker build step instead of runtime

RUN python -c 'from sentence_transformers import SentenceTransformer; SentenceTransformer("<model-name>")'
6reactions
ahmedshahriarcommented, Apr 1, 2022

Thanks for this @chanind

I added the path param and this saves the pre-trained model in the models directory

RUN python3 -c "from sentence_transformers import SentenceTransformer; model = SentenceTransformer('sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2'); model.save('models')"
Read more comments on GitHub >

github_iconTop Results From Across the Web

Manually Downloading Models in docker build with ...
Hi,. To avoid re-downloading the models every time my docker container is started, I want to manually download the models during building ......
Read more >
python - How to download and load ML model using sentence ...
With this example the model is going to be downloaded in the cache folder. ... A good approach is to use with multi-stages...
Read more >
Build and Run a Docker Container for your Machine Learning ...
The idea of this article is to do a quick and easy build of a Docker container with a simple machine learning model...
Read more >
Pretraining BERT from scratch on openwebtext data on a ...
I can hear you asking why do I need to build a docker image? ... It is possible to download the dataset within...
Read more >
huggingface/transformers-cpu - Docker Image
The second line of code downloads and caches the pretrained model used by the pipeline, while the third evaluates it on the given...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found