question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Failed and abandoned trials: indicating trial failures and and excluding certain parameter values from future trials

See original GitHub issue

I’m trying to optimize an evaluation function which consists of an optimization routine. For certain combinations of the parameters, the routine does not converge and results in an invalid result. I am currently using the GPEI optimizer and developer API with the SimpleExperiment class. I’d like to be able to indicate this when this happens and also somehow exclude those values from future trials, but wasn’t sure how to do this. I think #329 and #176 are related, but they don’t quite answer the question that I am looking for. I couldn’t really find any documentation about how to use mark_abandoned, which seems like what I need based on the explanation. Could I receive some suggestions for what to look at and how to implement my desired functionality? Thanks.

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:22 (12 by maintainers)

github_iconTop GitHub Comments

2reactions
ldworkincommented, Aug 26, 2020

@jangkj09 you’re right actually! I was getting confused about the difference between SimpleExperiment and regular Experiment. My example is really meant more for Experiment. SimpleExperiment does the trial evaluation automatically behind-the-scenes when you call eval, which makes it a bit harder to accomplish what you want. You probably want to use either regular Experiment or the Service API.

1reaction
jangkj09commented, Aug 28, 2020

Thanks for all your help!

Read more comments on GitHub >

github_iconTop Results From Across the Web

Factors associated with clinical trials that fail and opportunities ...
Failures can arise from a lack of efficacy, issues with safety, or a lack of funding to complete a trial, as well as...
Read more >
Cardiovascular Outcomes Assessment of ... - ClinicalTrials.gov
Cardiovascular Outcomes Assessment of the MitraClip Percutaneous Therapy for Heart Failure Patients With Functional Mitral Regurgitation (The COAPT Trial) ...
Read more >
E 9 Statistical Principles for Clinical Trials Step 5
trials is discussed, but is not a primary focus of this guidance. Selected principles and procedures related to data management or clinical trial...
Read more >
9-5.000 - Issues Related To Discovery, Trials, And Other ...
It is the obligation of federal prosecutors, in preparing for trial, to seek all exculpatory and impeachment information from all the members of...
Read more >
Key Insights from Two Decades of Clinical Trial Failures
To gain some insights, we reviewed AD clinical trials ... Third, we did not directly access the failed clinical trial data in detail...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found