Add the tracked value which can show the dimension collapse problem

We may follow the idea from LeCun’s paper and use the percentage of “big” singular values in SVD Technically we may put it inside the EmbeddingMetric

Another approach is to track how many principal components in PCA we need to keep 95% of accuracy. (What is the policy for the case when PCA improves accuracy?)

[dapladoc]

I like the proposed design. Just want to double-check: is it right that you want to call calc_topological_metrics somewhere nearby calc_retrieval_metrics in EmbeddingsMetrics class? If so, it’s okay.

Yes, calc_topolotical_metrics will be called near the calc_retrieval_metrics in EmbeddingMetrics.

The only thing that I would change is the current confusing signature. What about this? def count_meaningful_components(embeddings, examplained_variance_to_keep: Tuple[int, ...]) -> n_components Or you can suggest other ideas.

I’m ok with this naming.

[AlekseySh]

@dapladoc I like the proposed design. Just want to double-check: is it right that you want to call calc_topological_metrics somewhere nearby calc_retrieval_metrics in EmbeddingsMetrics class? If so, it’s okay.

The only thing that I would change is the current confusing signature. What about this? def count_meaningful_components(embeddings, examplained_variance_to_keep: Tuple[int, ...]) -> n_components Or you can suggest other ideas.

As for the issue for DirectCLR, then we need to add similar issues for the hundreds of other architectures, so, I would not do it.