Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Integration with the Hugging Face Hub

See original GitHub issue

Is your feature request related to a problem? Please describe. As I train models, I would like to easily be able to share them with other people and document them well. I would also like to be able to access other trained models from the community.

Describe the solution you’d like I would like to have an integration with the Hugging Face Hub (disclaimer: I’m a member of the OS team there). I would like to be able to do model.push_to_hub("osanseviero/my_vae") and get a model directly in the Hub. Some of the benefits of sharing models through the Hub:

  • versioning, commit history and diffs
  • repos provide useful metadata about their tasks, languages, metrics, etc that make them discoverable
  • multiple features from TensorBoard visualizations, leaderboards, and more

Issue Analytics

  • State:closed
  • Created a year ago
  • Reactions:2
  • Comments:11 (6 by maintainers)

github_iconTop GitHub Comments

clementchadebeccommented, Jul 5, 2022

Hi @osanseviero, Thank you for requesting this feature. It was added in #28 and we can now share and download models though the HuggingFace Hub! To do so you need:

  • a valid HuggingFace account
  • the package huggingface_hub installed in your virtual env. If not you can install it with
$ python -m pip install huggingface_hub
  • to be logged in to your HuggingFace account using
$ huggingface-cli login

Uploading a model to the Hub

Any Pythae model can be easily uploaded using the method push_to_hf_hub

>>> my_vae_model.push_to_hf_hub(hf_hub_path="your_hf_username/your_hf_hub_repo")

Note: If your_hf_hub_repo already exists and is not empty, files will be overridden. In case, the repo your_hf_hub_repo does not exist, a folder having the same name will be created.

Downloading models from the Hub

Equivalently, you can download or reload any Pythae model directly from the Hub using the method load_from_hf_hub

>>> from pythae.models import AutoModel
>>> my_downloaded_vae = AutoModel.load_from_hf_hub(hf_hub_path="path_to_hf_repo")
osansevierocommented, Jul 7, 2022

cc @nateraw @nimaboscarino

Yes, the way we add metadata is by creating a file with yaml metadata at the top. Having the small snippet you suggested would work great. huggingface_hub has a couple of utilities to make updating the metadata very very easy. For example, you can use metadata_update to just add the tag. We also have a couple of nice utilities being added to huggingface_hub for easier programmatic model card creation ( being merged right now), but the existing tools should work well already. Let us know if you have any feedback!

Read more comments on GitHub >

github_iconTop Results From Across the Web

Hugging Face Hub documentation
The Hugging Face Hub is a platform with over 60K models, 6K datasets, and 6K demos in which people can easily collaborate in...
Read more >
Integrations with the Hugging Face Hub
To make your life even easier, Gradio integrates directly with Hugging Face Hub and Hugging Face Spaces. You can load demos from the...
Read more >
Uploading models
To upload models to the Hub, you'll need to create an account at Hugging Face. Models on the Hub are Git-based repositories, which...
Read more >
Quick start
The Hugging Face Hub is the go-to place for sharing machine learning models, demos, datasets, and metrics. huggingface_hub library helps you interact with ......
Read more >
Getting Started with Repositories
Using the Hub's web interface you can easily create repositories, add files (even large ones!), explore models, visualize diffs, and much more. There...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found