Difficulty Linking Charts While Abusing Altair.

Let me start by explaining my madness:

I am working on a gameplaying algorithm. In one of the steps, the algorithm makes a decision to load a specific game state. And in the analysis step – which I am doing right now – I have the details of how many times an individual state has been selected and a screenshot that goes along with that state.

Wanting to learn Altair, because it is the best thing invented since toasted bread, I decided to create a simple visualization:

1. Have a bar chart that shows how many times each state has been selected.
2. When I click on any bar, display the image associated with that state somewhere.

I soon realized that Altair cannot display images. I was a bit bummed, but I REALLY wanted this. Then it hit me. Altair COULD display scatter plots, which is simply a series of colored points. What was an image, if not a series of colored points?

(Portion of the code to create this)

A data frame that contained (X, Y, Color) later I had it working. Now I just needed to connect it to the bar chart. And this where I am at. (Working with downscaled images for performance)

Let me explain the behavior:

1. On the left, I am showing how many times each state has been selected as a bar chart, on the right our downsampled image scatter plot. I think there are hundreds of bars being drawn on top of each other, which might be a problem.
2. I am drawing ALL the images on top of each other with .5 opacity.
3. When I click on one, it filters based on the index. (This is why it goes mostly black initially.)
4. When I click on whitespace all are back with a different z-fighting order.
5. When I hover over both the images and the bar charts I can see the “index” as the tooltip.
6. When I click on the bar chart all the images disappear.

#3 shows me that filtering is working. #5 shows me that the index selection is correct.

However, I couldn’t get it to work beyond what you see here. The code that displays the charts are below. (If you need to see the DF creation. On the high level, it is similar to the earlier example but joined on all the values, per image.)

clicker = alt.selection_single(empty="all", fields=['index'])

chart = alt.Chart(images_pd).mark_bar().encode(
    # Couldn't get it to work with mod(index):Q
    x='index:Q',
    y='explore_counts:Q',
    tooltip=["index"]
).add_selection(
    clicker
)

img_chart = alt.Chart(images_pd).mark_square(size=400, opacity=.5).encode(
    x="x:Q",
    y="y:Q",
    color=alt.Color('color', scale=None),
    tooltip=["index"]
).transform_filter(
    clicker
).add_selection(
    clicker
)

chart | img_chart`

Any help would be appreciated, and thank you very much for this library. I find it incredibly elegant and powerful.

[jakevdp]

Vega-Lite currently doesn’t support image marks, but there’s an open issue discussing it: https://github.com/vega/vega-lite/issues/3758

One issue with scalability is the fact that currently each image pixel is stored in a row of the dataset, along with a copy of all the metadata associated with the image. You can get around that by storing the image data in a 2D array within a single entry of the data table, and then use a flatten transform within the chart spec to pull it out. That should help push the limit of the pixel resolution in your heatmap.

[mattijn]

Seems similar to https://altair-viz.github.io/gallery/select_detail.html

Using init in your selection variable it’s possible to define a start selection and with empty='none' nothing is drawn when nothing is selected.

I think you should be able to extract the needed logic from here.

import altair as alt
import pandas as pd
import numpy as np

np.random.seed(0)

n_objects = 20
n_times = 50

# Create one (x, y) pair of metadata per object
locations = pd.DataFrame({
    'id': range(n_objects),
    'x': np.random.randn(n_objects),
    'y': np.random.randn(n_objects)
})

# Create a 50-element time-series for each object
timeseries = pd.DataFrame(np.random.randn(n_times, n_objects).cumsum(0),
                          columns=locations['id'],
                          index=pd.RangeIndex(0, n_times, name='time'))

# Melt the wide-form timeseries into a long-form view
timeseries = timeseries.reset_index().melt('time')

# Merge the (x, y) metadata into the long-form view
timeseries['id'] = timeseries['id'].astype(int)  # make merge not complain
data = pd.merge(timeseries, locations, on='id')

data.head()

<div>

	time	id	value	x	y
0	0	0	-1.048553	1.764052	-2.55299
1	1	0	-1.721013	1.764052	-2.55299
2	2	0	-2.886163	1.764052	-2.55299
3	3	0	-1.003013	1.764052	-2.55299
4	4	0	-0.626587	1.764052	-2.55299

</div>

# Data is prepared, now make a chart
selector = alt.selection_single(empty='none', fields=['id'], init={'id':8})

base = alt.Chart(data).properties(
    width=250,
    height=250
).add_selection(selector)

bars = base.mark_bar().encode(
    x='id:O',
    y='mean(y)',
    color=alt.condition(selector, 'id:O', alt.value('lightgray'), legend=None),
)

scatter = base.mark_point().encode(
    x='time',
    y=alt.Y('value', scale=alt.Scale(domain=(-15, 15))),
    color=alt.Color('id:O', legend=None)
).transform_filter(
    selector
)

bars | scatter