Enable registration or passing of a custom triplet loss function

In Python, Ivis.__init__ accepts a distance: str keyword argument, which sets from a dictionary a predefined triplet loss function for that distance metric. Currently, one of the ways to provide a custom distance function is to monkeypatch the ivis.nn.losses.get_loss_functions. Other ways to accomplish the same are even messier from the perspectives of usage and implementation.

The nature of dimensionality reduction, especially when dealing with one-hot-encoded categorical features, sometimes requires custom ways to calculate loss. Under the hood, ivis has the ability to enable custom loss functions, but any such offerings need to be implemented in a clean and API-idiomatic manner.

---

A custom distance function requires its own triplet loss implementation. Ivis.__init__ could support an additional keyword argument (e.g. triplet_loss: Callable[..., ...] = ...) for users to be able to pass their own.

Alternatively, it could simply be passed inside the existing distance kwarg, with its signature changing to distance: Union[str, Callable[..., ...]].

Another way would be to make the losses dictionary built by ivis.nn.losses.get_loss_functions a module-level loss function registrar.

Additionally, docs and examples need to be updated on how to correctly implement a custom loss function. With all currently available distance metrics, the triplet loss implementation follows a very similar pattern, and should not be too daunting to attempt to implement.

[Szubie]

Hi, since this issue was opened there have been some relevant changes to ivis.

1. It is now possible to pass custom loss functions to ivis as requested in this issue. The simplest way to do this is to pass the desired callable to the distance kwarg when constructing the ivis object. (Alternatively you can register a loss function globally using the register_loss function in the ivis.nn.losses module and reference by name string).

2. The neighbour retrieval component of ivis is now swappable to any arbitrary implementation by providing an instance of the collections.abc.Sequence class to the neighbour_matrix kwarg. To implement the methods of the Sequence class, this object must have a __len__ method that matches the number of rows in dataset, and a __getitem__ method that returns a list of neighbour indices for the row with the given index. Check the ivis.data.neighbour_retrieval module for examples.

   • For example, the neighbour_matrix argument could be a 2D numpy array (rows x neighbours).

Hopefully these changes make it easier to use your own classes to customize the behaviour of ivis.

[mihajenko]

I misunderstood the dependency between the KNN metrics and the triplet loss metrics, thank you for clarifying. Apart from possible serialization issues, I don’t see much in the way of implementing the custom triplet loss feature.

When someone has a concrete usecase for it, I’ll focus on this issue again.

The feature with the bigger priority for me would actually be a swappable KNN index. To that end, I am integrating the NGT indexer into ivis, because it supports the metric that I need. Feature branch is here: https://github.com/mihajenko/ivis/tree/feature/ngt-backend