MiniLM: releasing all models?

Hi there,

First of all: great work on distilling a strong teacher into a well performing student and eliminating the issue of parameter size discrepancy in teacher-student models! I am always happy to see smaller, usable models.

I was wondering if you plan to release the Small MiniLM model (L6xH384). It says We release the uncased 12-layer and 6-layer MiniLM models with 384 hidden size [...], but I can only find the link to the 12-layer model.

Thanks so much

[nreimers]

Hi @WenhuiWang0824

great, thank you for the great work and releasing the model. The MiniLMv1 models work great for bi-encoders and cross-encoders, so I’m eager to test the v2 models.

It would be great if the models could also be added to the huggingface model hub: https://huggingface.co/microsoft

This would make it easy to load and use the models.

Let me know if you need help putting the models on the hub.

[wenhui0924]

Hi @volker42maru and @maksymbevza,

We have released the monolingual and multilingual minilmv2 models distilled from different teachers. Please find the model links in the MiniLM folder.

Thanks