Store user token in huggingface home folder

At the moment when the user uses login() to set its User Access Token, the token is stored in the file ~/.huggingface/token.

This path is currently not configurable. The right behavior would be to reuse “HF home folder”. By order of priority, this means the token would be stored under:

# if HF_HOME is set
$HF_HOME/token

# otherwise if XDG_CACHE_HOME is set (unix)
$XDG_CACHE_HOME/huggingface/token

# otherwise in default folder
~/.cache/huggingface/token

In the process of making this change we should take care of not breaking anything that currently exists. As I see it, we could:

1. On HfFolder.save_token(...): store the token in HF_HOME/token.
2. On HfFolder.delete_token(): delete from ~/.huggingface/token and HF_HOME/token
3. On HfFolder.get_token(): read from HF_HOME/token if it exists. Otherwise read from ~/.huggingface/token, save under HF_HOME/token and return the token.

In all 3 cases, raise a warning “hey, you should delete ~/.huggingface/token as this location is deprecated”. This would ensure that with time all users will move to the new token path. I prefer the warning message instead of just deleting the old token path in case the user has multiple virtualenvs with multiple versions of huggingface_hub. We don’t want the user to be forced to login again each time one uses an old version of hfh.

Let’s take care of the consequences on downstream libraries as well (cc @sgugger @LysandreJik @osanseviero)

[Wauplin]

Still fixed it in the end

[julien-c]

is it worth the hassle if no-one has complained about it? (no strong opinion, just asking)