General event support

Proposal

I think that OSMD integration/adoption could be enhanced by creating a general event (not just DOM) system that callers can hook into.

Hopefully these contrived examples are illustrative of the possibilities:

type OSMDEvent<P, M = any> = { id: number; metadata: M; payload: P };

type CursorChangeEvent = OSMDEvent<{
  cursor: Cursor;
}>;

enum GraphicType {
  Note,
  Lyric,
}

type GraphicEvent = OSMDEvent<
  | { type: GraphicType.Note; note: Note }
  | { type: GraphicType.Lyric; lyric: Lyric, syllabic: boolean, text: string }
>;

// Non-DOM event: Vibrate a device every time the cursor changes.
osmd.on("cursorchange", (event: CursorChangeEvent) => {
  makeHapticFeedback(HapticFeedbackIntensity.Gentle, Duration.ms(10));
});

// Color a note head whenever it's hovered.
osmd.on("graphichover", (event: GraphicEvent) => {
  if (event.payload.type === GraphicType.Note) {
    const { note } = event.payload;
    if (noteColorer.currentlyColoredNote && noteColorer.currentlyColoredNote !== note) {
      noteColorer.restoreOriginalColor(noteColorer.currentlyColoredNote);
    }
    noteColorer.color(note, 'osmd-orange');
  }
});

// Announce the entire word of a lyric when it's clicked.
osmd.on("graphicclick", (event: GraphicEvent) => {
  if (event.payload.type === GraphicType.Lyric) {
    const { lyric, syllabic, text } = event.payload;
    const word = syllabic ? [text, ...getRemainingSyllablesAfter(lyric)].map(withoutTrailingHyphens).join('') : text;
    if (screenreader.canPronounce(word)) {
      screenreader.announce(word);
    }
  }

Discussion

I think this can be done in a way that is not invasive and can be introduced gradually.

A possible solution could be to create an event bus that allows OSMD callers to hook into. Internally, the event bus would maintain a mapping of event names to arrays of listeners. Within the OSMD codebase, this event bus would be used to dispatch events with payloads that correspond to the event name.

One of the drawbacks of this approach is that it is the caller’s responsibility for cleaning up listeners, since OSMD would not know nor care about the context outside of its purview. Perhaps there’s something clever that can be done with WeakRef, but I’m unsure.

Another benefit of this approach is that it may surface refactoring opportunities within the OSMD codebase. For example, if there’s a cursorchange being dispatched from several methods, it might suggest that there’s a common behavior that can be abstracted out amongst them. It also might suggest that the event dispatching is done in the wrong place as well as a bunch of other possibilities.

Related Issues

#262 0xfe/vexflow#371

[jaredjj3]

@bukaznik, I’ve done a soft release of stringsync but anyone can access it. I have 1 transcription available at https://stringsync.com/n/NRZF7Cjm. You may have to hard refresh if it’s not loading.

Feel free to add stringsync to https://opensheetmusicdisplay.org/showcases/

[jaredjj3]

I’d like to make a progress update. I was able to make a working version of the event system, but I had to leverage the concept of iterator snapshots described in https://github.com/opensheetmusicdisplay/opensheetmusicdisplay/issues/139#issuecomment-903201412. Anyone that wants to adopt this event system directly would also need that use that solution as well.

The core implementation is contained within the SVGEventProxy (link is pinned at rough solution commit).

One of my goals before trying to “publish” this as a community example is to make the solution reasonably maintainable. In the code’s current state, it is already becoming difficult to conditionally dispatch certain events. For example, if the user is dragging a cursor, we don’t want to dispatch selectionupdated events. Right now, I’m imperatively juggling state within the SVGEventProxy and I can see this easily ballooning into something unruly. My plan is to create state machine(s) that would allow me to add more events in a declarative manner. I think this will solve the maintainability issue. I plan to use the xstate library.

Another goal is to make this solution resource efficient. I think I’ve generally accomplished this (throttling mousemove and touchmove events, installing 1 click handler per (event type, svg element), etc.), but I think there are micro optimizations that I can still make. For example, when determining if a cursor is being “hit” (on hovered, mousedown, touchdown, etc.), I suspect that calling getBoundingClientRect may be expensive in some contexts. To optimize, I could cache the hit and use it to shortcut the next calculation. All in all, I want to limit querying the DOM for events that may be dispatched frequently.

Here is a demo of me dispatching mousemove, cursordragstarted, and cursordragended events:

https://user-images.githubusercontent.com/19232300/131683654-1706861a-84c3-43e4-8548-429906b12be8.mov