Drop High/Drop Low are not working correctly

I get a different answer depending on whether I fit just the work comp LOB vs all LOBs and then slice work comp. These approaches should yield the same answer, but they don’t:

import chainladder as cl
clrd = cl.load_sample('clrd').groupby('LOB')['CumPaidLoss'].sum()
# Passes
assert cl.Development(n_periods=5).fit(clrd).ldf_.loc['wkcomp'] == cl.Development(n_periods=5).fit(clrd.loc['wkcomp']).ldf_
# Fails
assert cl.Development(drop_low=2).fit(clrd).ldf_.loc['wkcomp'] == cl.Development(drop_low=2).fit(clrd.loc['wkcomp']).ldf_

I don’t think this ever worked. The reason is that we only have one 2D (origin x development) matrix of the entries to be dropped. This matrix is shared across all triangles. When dropping using n_periods, drop_valutation, drop_origin, etc, this is fine. But dropping based on link-ratio values it is not.

[hoverat]

I was with some non-actuaries this weekend, and I expressed some verbal excitement when I saw this issue is being fixed. Even with their desire to join in, the excitement of a working xHiLow average was not easily transferable. Thanks for doing this!

[henrydingliu]

In the code, drop and drop_high/drop_low are applied in parallel. (interestingly enough, drop_above/drop_below are applied after.) Not a difficult thing to put them in any order, but I wonder if the better approach is to allow the practitioner to apply the precise order they want via a pipeline. Unfortunately it looks like Development transformers are memoryless at the moment (see example below).

genins = cl.load_sample("genins")
genins_dev_1 =cl.Development(n_periods = 2).fit_transform(genins)
genins_dev_2 =cl.Development(n_periods = 4).fit_transform(genins_dev_1)