conceptual question about the embeddings

Hello there! and thanks for this package. It is really super fast and efficient.

I just have a conceptual question about the models that are available in sentence-transformers. Are they trained for a particular task or they come from big unsupervised masked language models (say BERT) trained on massive amounts of text?

For instance, if I were to train a BERT masked language model from scratch using huggingface I would also obtain embeddings at the sentence level. Would these embeddings be similar to the ones available in sentence-transformers? The reason I am asking is because I want to trained a masked language model on my highly specialized corpus of text.

Thanks!

[nreimers]

Yes

[nreimers]

Only one