Text labels in scatterplot not behaving as expected

I am trying to create a scatterplot visualization of the embedding space for my vision model by looking at the t-SNE projections of a few classes. As far as the plotting goes, I seem to have everything wired up. In this attached image, you can see that I’m looking at a plot of dog and cat embeddings. The problem I am having is that the ‘textlabels’ that I have supplied the plot function are not showing up. My goal is to show the image id for the given data point, so that I can dive deeper into data points that are not where I’d expect them to be (e.g. a dog image deep within the cat cluster). I would expect the mouseover to show either the class and my supplied label, or at the very least, my supplied label. In the current implementation, it’s instead only showing the class.

The code supporting my visdom plot:

print('Computing t-SNE')
# This will produce the (N, 2) coordinate matrix
vis_pos = bh_sne(all_vecs)
# This is the (N,) matrix of labels for each point
vis_lab = numpy.concatenate([numpy.repeat(i+1, v.shape[0]) for i, v in enumerate(q_vecs)])
print('Done')
print('Plotting...')
opts = {
    'legend': q_list,
    'title': args.plot,
    # This is a python list of string ids for each image (size N)
    'textlabels': all_ids,
    'mode': 'markers+text'
}
print('vis_pos', vis_pos.shape)
print('vis_lab', vis_lab.shape)
print('all_ids', len(all_ids))
vis.scatter(X=vis_pos, Y=vis_lab, win=args.plot, opts=opts)

Sample Output:

Plotting...
vis_pos (2000, 2)
vis_lab (2000,)
all_ids 2000

Both my client and server version of visdom is 0.1.7

I was looking into the source code (master), and there appears to be a few bugs in at least the client code. I’ll try to outline what I think I’m seeing, but my workaround also doesn’t work. In this block: https://github.com/facebookresearch/visdom/blob/4d9f7c0e6a896943707fd255e0d46bc6798a42e3/py/__init__.py#L838-L872

It appears to be iterating over each class, and preparing the markers separately for the data for each respective class. The problem is, the function doesn’t also select the relevant textlabels for the given class on line 854.

The apparent workaround to this, just to see if I could get the textlabels to show was to set all of the points to the same class, which should cause the supplied labels to again align with the points. The problem is, I still don’t see my supplied labels, as shown.

Is there a setting that I am missing, or a known bug for this? I’m also happy to dig further into this if someone can point me in the rough direction of the server code that’s making the plotly calls.

Thanks

[JackUrb]

@dizcza yeah it should technically be possible, you need to change opts['mode'] to get the behavior you want. I believe opts['mode'] = 'lines+markers+text' will display the text on the markers without hover, opts['mode'] = 'lines+markers' will only display the text on hover. You do need to use textlabels in both cases.

[cympfh]

↑ 0.1.7 is too old to use textlabels. BTW I want .__version__.