is kneed picking the right knee point?

Hi Kevin,

Let me first say thanks for your package!

I am however wondering whether it picks the proper knee point. kneed was installed using conda, showing below version kneed 0.2.4 py_0 conda-forge

I’m running the following code:

y = [7304.988411743468, 6978.98441315824, 6666.605130591402, 6463.195596457663, 6326.525947962969, 
     6048.793513285322, 6032.793220988797, 5762.013547650833, 5742.773572835342, 5398.219072974353, 
     5256.840796466522, 5226.976690998346, 5001.718982839869, 4941.984506219156, 4854.238126768628, 
     4734.606344213364, 4558.749543289275, 4491.096245408976, 4411.612468308233, 4333.007985566374, 
     4234.633219330786, 4139.10266640919, 4056.8041571434956, 4022.4882313410208, 3867.9649688469103, 
     3808.266172761056, 3745.267272596804, 3692.343525689731, 3645.5533571744386, 3618.2781512814176, 
     3574.26074141688, 3504.3061262646543, 3452.444673732173, 3401.19897729189, 3382.3740348889764, 
     3340.6702550205196, 3301.0814510684318, 3247.5885929108044, 3190.270755323219, 3179.9905812848137, 
     3154.2367011478286, 3089.5396073964585, 3045.61707926626, 2988.993953177785, 2993.614218064459, 
     2941.346229778838, 2875.5955684762366, 2866.3253584487247, 2834.117714931289, 2785.1456843776896, 
     2759.6514682361576, 2763.2024159338034, 2720.1356002598905, 2660.140799793623, 2690.2175242045923, 
     2635.7118932237527, 2632.9222293329244, 2574.6268292686323, 2555.965416634073, 2545.7190837261787, 
     2513.381871491499, 2491.5685394975612, 2496.0498636812163, 2466.450734239057, 2442.7208947876484, 
     2420.5347333116115, 2381.537840563225, 2388.0917624044428, 2340.6133232506804, 2335.0286400024625, 
     2318.927311833302, 2319.0470174461234, 2308.234710680705, 2262.22706400162, 2235.7819112049838, 
     2259.270039523929, 2221.0453886756854, 2202.6929974205277, 2184.288565745161, 2170.0699701041776, 
     2160.0469316534904, 2127.6818252628573, 2134.731718758979, 2101.962979423362, 2101.441656971703, 
     2066.4026551229113, 2074.2546618976407, 2063.6767099234867, 2048.1153581337485, 2031.8747775400475]
x = list(range(90))

kneedle = kneed.KneeLocator(x,y,S=1,curve='convex', direction='decreasing')
print(kneedle.knee)
kneedle.plot_knee_normalized()
kneedle.plot_knee()

If you look at the resulting plots below, I would expect the knee point to be more to the left, basically where you see the max of the difference curve (around 0.3 on the normalized plot in my case). At least that’s my understanding of the kneedle algorithm… I tried with different values of S (smaller and bigger), but can’t see any changes here. BTW, Sensitivity is a bit counter intuitive, the smaller S the more coarse the algorithm, maybe it’s an idea to put it in the denominator of the formula and/or rename it. Haven’t looked into the code yet, but still planning to do so…

–Peter

[Peterdha]

Hi @arvkevi , have been looking a bit closer at the code, I can’t see anything wrong with your implementation, however I do believe the algorithm is flawed. In the case where there is some jitter on the curve we want to find a knee for, it means we have a lot of local maxima and minima. This results in always resetting the threshold, resulting in outputting as a knee the last local max that was found. My guess is the smoothing step is the culprit here, as a choice for a spline will always pass through the datapoints themselves. In my case that means the jitter stays in! So I hacked “exponentially weighted moving average” in, as mentioned in the article to be an alternative for splines, works like a charm now 😃 ! As illustrated below. Using the solution from Divakar on https://stackoverflow.com/questions/42869495/numpy-version-of-exponential-weighted-moving-average-equivalent-to-pandas-ewm

However the S-factor doesn’t do a whole lot for me…

[arvkevi]

fixed by #26