Multichannel display

We’ve previously decided that #1130 does not directly include modifications to the display module, and that those functions should remain mono-only. However, this doesn’t rule out the possibility of adding new functionality to support multichannel visualizations easily, and I think this would be quite helpful in general.

Describe the solution you’d like

This is easiest to consider starting from wave plots. In the common case of 2-channel signals, I think it would be helpful to have a plot that renders a joined pair of subplots with one channel each and shared axes, like you might see in a DAW.

Here’s some prototype code that does this already:

fig = plt.figure(figsize=(10, 5), constrained_layout=False)

grid = fig.add_gridspec(2, 1, wspace=0, hspace=0)
axes = grid.subplots(sharex=True, sharey=True)

librosa.display.waveshow(y[0], sr=sr, ax=axes[0])
librosa.display.waveshow(y[1], sr=sr, ax=axes[1])
axes[0].label_outer()
axes[0].set(xlabel=None)
fig.show()

Which produces the following image with the trumpet example:

Note that this code would break the current pattern of display functions accepting target axes, since it will need to create an axes for each channel. I think this is fine, and we can still accept fig as a target or create a new one as needed.

We also have to do a little bit of cleanup here to hide the xlabel and ticks. (Side note: maybe waveshow and specshow should have options to not set axis labels.)

This approach would easily generalize to C>2 channels, though if we have higher order arrangements we might get into trouble and have to flatten the array down in advance.

Some parameters we should expose (with defaults):

• wspace,hspace=0: spacing between subplots. I think most people would expect vertical packing as i’ve done in this example, but it should be configurable.
• sharex,sharey=True: subplot axis coupling. This seems generally desirable, but maybe there are use cases where it would not be helpful.
• waveshow parameters (passthrough)

The function would then return, I guess, the grid object? This seems to be how seaborn does it for a similar use-case, and they seem pretty on-the-ball.

We could do similar things for specshow, though we might not want to default to dense packing there because specshow plots don’t have the benefit of negative visual space like waves do.

Describe alternatives you’ve considered We could also just punt this and leave people to their own devices, but that seems like a disservice.

[bmcfee]

Just doing a little more hackery on this after browsing some common DAW interfaces. Curious how folks feel about using background (axes face) coloring for different channels?

Here’s a quick hack:

fig = plt.figure(figsize=(10, 2), constrained_layout=False)

grid = fig.add_gridspec(2, 1, wspace=0, hspace=0)
axes = grid.subplots(sharex=True, sharey=True)

librosa.display.waveshow(y[0], sr=sr, ax=axes[0], color='#3d3d3d')
librosa.display.waveshow(y[1], sr=sr, ax=axes[1], color='#3d3d3d')
axes[0].label_outer()
props = axes[0]._get_lines.prop_cycler
for ax, col in zip(axes, props):
    ax.set_facecolor(col['color'])

axes[0].set(xlabel=None)
axes[0].set(xlim=[0, librosa.get_duration(y=y, sr=sr)])

Very different from our current aesthetic, and could surely be optimized, but it seems like an interesting direction to try out.

[bmcfee]

Thinking this one over, I don’t think we actually need to accept a gridspec if we can take an axes array. Once the grid is specified and constructed, the rest of the plotting (wave, spectrogram, or otherwise) should not need to know anything about the overall figure geometry and can operate independently on each axes object.

This would make it relatively easy to do something like the image in https://github.com/librosa/librosa/issues/1370#issuecomment-1148787465 by constructing axes with a gridspec, and then slicing the axes array axes[0, :], axes[1, :], etc as input to the multi-plot function(s).

For simple cases (e.g. only waves) we can automate this and accept gridspec_kw as a parameter to set things up. For more complex cases, having the following seems not so bad:

grid = fig.add_gridspec(2, 2, wspace=0, hspace=0, height_ratios=(0.25, 0.75))
axes = grid.subplots()

# Plot waves on the first row axes[0, :] --> map out to waveshow if y is not None
librosa.display.multishow(y=y, axes=axes[0, :], sharex=True, sharey='col')

# Plot chromas on the second row axes[1, :] --> map out to specshow if data is not None
librosa.display.multishow(data=chroma, axes=axes[1, :], sharex=True, sharey='col')

and of course, the first two lines could also be done compactly via

fig, axes = plt.subplots(..., gridspec_kw=dict(wspace=0, hspace=0, ...))