RQA and Smith-Waterman

Description

(Forking this issue off from #741 , since it’s pretty different from the initial idea there. Tagging @ctralie for input though.)

librosa does not currently implement much in the way of recurrence quantification analysis, beyond standard DTW. It would be easy to provide some basic functions for recurrence post-processing, such as the L/S/Q methods described by Serra et al., 2009, and Smith-Waterman.

The L-method simply accumulates diagonals, and resets the counter whenever there’s a gap (R[i,j] = 0).

The S-method accumulates diagonals, but allows for gaps of length 1 by considering the max over the immediate diagonal and knight’s moves away from R[i, j].

The Q method is closely related to Smith-Waterman, with the following modifications:

• only gaps of length 1 are directly computed
• instead of diagonal/horizontal/vertical, the gaps are diagonal and knight’s moves
• different gap penalties are applied depending on whether an entry introduces a new gap or extends an existing one.

While Q and SW are related, I don’t see a clean way to implement both with the same underlying function. I think it would be simpler to have two independent functions for these. Q and S can be simply implemented though: as noted by Serra et al., when the gap penalties become large, the Q method simplifies to the S method (all gaps instigate a reset). Likewise, if we make knight’s moves a toggle, then it’s easy to restrict Q down to L (with large penalties and knight moves disabled).

As mentioned in the previous thread, I’ve prototyped these already with numba, and they’re pretty fast. The only remaining challenges are:

1. supporting sparse input
2. making a generic API to encapsulate all of the above methods
3. generalize to accumulate link strength rather than counting steps. With binary input, this recovers the original implementation.

[bmcfee]

Nice! That phase transition between 10 and 20 is interesting; I wonder what’s all about.

You might also find jakevdp’s post on non-uniform FFT implementations interesting: https://jakevdp.github.io/blog/2015/02/24/optimizing-python-with-numpy-and-numba/ as it touches on some of these issues.

[ctralie]

Great point! Thanks for keeping me honest. It looks more like a factor of 3:

Anyway, I’m excited about two things here

1. How simple numba is, and how much better it is than raw Python (never used it before)
2. That there’s still a 3x factor to be had for people like me who are doing large scale experiments

I’ve been having an absolute nightmare getting cython to work cross platform in another project, so I’ll definitely look into numba