Abandoned arms and trials are not excluded from modeling data

hey guys, thanks for offering such a great lib!

In case I found the data of an already completed arm is invalid, how can I remove it to make sure the model prediction is not effected by this invalid arm?

I tried to abandon this arm like experiment.trials[0].mark_arm_abandoned(arm_name='0_0') but seems the abandoned arm is still taken into account in model fit process.

Thanks in advance 😃

[lena-kashtelyan]

Cool, then you can use the Dev API with regular Experiment and abandoned trials should work as expected. Let us know if you run into any other issues!

[ugurmengilli]

Is there a reason why you need SimpleExperiment specifically, or would you be able to switch to the Service API, where the abandoned trials functionality is properly supported?

There isn’t, actually. I started using SimpleExperiment because an Experiment needs proper coordination of Trial, Runner, and Metric (was very hard to track at first), and the Service API is hiding the details of underlying steps (I needed decent control over it to match the theory and implementation of my work). I could recently switch to Experiment after discovering the multi-objective opt tutorial and some hands-on experience. I’m keeping the previous one in case I need a quick implementation, though I didn’t know that you would completely drop it.