Is it possible to train CosineSimilarityLoss without labels?

I’d like to rebuild a model from a certain paper that minimizes 1 - CosineSimilarity(a, b), but it seems to me that the implementation of CosineSimilarityLoss in this repo and the usage in the examples always require a label (i.e. -1 or 1). Is it possible to train the CosineSimilarityLoss loss without labels, providing only positive/similar example pairs?

[nreimers]

Hi @olastor The CosineSimilarityLoss requires a continous label, e.g. from -1 to 1 or from 0 to 1.

If you have only positive pairs, you can construct negative pairs by randomly selecting two pairs. This usually gives you a negative pair. Then you can train with ConstrativeLoss.

Else have a look at triplet loss.

Best Nils Reimers

[datistiquo]

I think if you have labels 0 and 1, then cosine sim schould also work.

But this is eaxatcly what you also do in the paper, right? So I am a bit unsure what your “should work” means? Maybe you use the gold scores a s “labels”?

From the paper:

Regression Objective Function. The cosinesimilarity between the two sentence embeddings u and v is computed (Figure 2). We use meansquared-error loss as the objective function.