Fine-Tuning STS and semantic search multilingual HF transformer

Hi,

For the need of exporting transformer model to ONNX format for inference, I use a multilingual sentence-transformer model based on the HF transformers library (separate tokenizer and model + mean pooling layer) for semantic text similarity and semantic search.

Is it possible to fine-tune this model with # https://github.com/UKPLab/sentence-transformers/tree/master/examples/training/sts/training_stsbenchmark_continue_training.py despite not using the sentence-transformer library directly?

If not is there a workaround to enable to do this? (like fine-tune sentence-transformer model then split tokenizer/model before ONNX export)

Thanks!

[nreimers]

Correct.

Number of examples depends on how complex is your domain. With a simple and narrow domain 1k examples are helpful.

With a broad domain that spans over basically all topics (physics, math, sports, gaming, dating, programming,…) you need a lot more examples. If you don’t have examples for eg math, the model will not work that well for math queries.

[Matthieu-Tinycoaching]

That sounds great @nreimers! So, regular transformer model without mean pooling?

Could you give me an idea of how many minimum training examples would be needed to gain from fine-tuning pre-trained multilingual sentence-transformer for STS and semantic search?