Are umap transformations non-deterministic?

I am trying to use umap to preprocess some data, and I’ve noticed that the same vector gives a different result according to the number of rows that is being passed to the transformation.

i.e the same row vector A outputs different vector according to the shape of the data (# of rows) being transformed.

# fit umap to data X 
reducer = umap.UMAP().fit(X)
# transform X using reducer 
embedding = reducer.transform(X)
# get subset of X to transform 
embedding_sub = reducer.transform(X[:100,:])
# => I was assuming embedding_sub == embedding[:100, :]
# => but that wasn't the case

[dataist]

Hey all, I thought I’d add some analysis to this discussion via colab here.

There I do quick quantification of the randomness introduced and explore its impact on some downstream tasks. You can find conclusions below (also in notebook):

We see cluster assignment can be super unstable as a result of the noise introduced by umap’s transform even when it is the exact data used for embed training. Obviously, if your downstream task maps to what we illustrate above you may be in trouble and your downstream tasks heavily impacted by the stochastic nature of the umap transform…

Im curious if there are any parameters we can add to the underlying knn search to help reduce the scale of the stochastic noise it introduces (even at computational expense)?

Separately, I would argue that the current implementation’s aggressive memoization used by the transform function could obfuscate downstream problems. If its stochastic nature was evident in all scenarios it would be more obvious and help highlight when it was a source of problems like we illustrate here.

I also did some additional parameter exploration on how each may help to mitigate some of the scale of the issue (not illustrated here):

        n_neighbors: no impact
        n_components: no impact
        metric: no impact
        n_epochs: some (30%) improvement as we go over 1000
        learning_rate: some (20%) improvement as we get around 0.5
        init: no impact 
        min_dist: some impact in concert with spread 
        spread: some positive impact by setting this to np.std of X_train 
        set_op_mix_ratio: seems best to keep at 1
        local_connectivity: no impact,
        transform_queue_size: no impact
        random_state: no impact
        transform_seed: no impact

[apbard]

I would like to add to @dataist’s experiments that we get different result for the same point also if:

• data are shuffled
• there are duplicate points (in this case it seems that just the first occurence for each duplicate assume a different embedding)