Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

New Evaluation: Biology

See original GitHub issue

We would like to find or create an eval dataset that tests biological and medical knowledge. This dataset for evaluating questions about COVID is a good place to start.

  • Data processing code implemented
  • Evaluation implemented

The evaluation code should be modeled after the interface in lm_eval/ and the example of the BoolQ task in lm_eval/tasks/

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:8 (8 by maintainers)

github_iconTop GitHub Comments

leogao2commented, Feb 6, 2021

We shouldn’t be having these big omnibus issues; for each dataset, please make an issue and add it to the project board. I’ve made separate issues for the following:

Pubmed: #125 emrQA: #115 BioMRC: #126 HeadQA: #127 BioASQ: #114

nicholaskrosscommented, Jan 8, 2021

Perfect, thanks!

Read more comments on GitHub >

github_iconTop Results From Across the Web

New Method Could Allow Better Evaluation of Cells
A team led by Northwestern Engineering's Madhav Mani developed a new method that offers a general and direct improvement to a broad class...
Read more >
On the optimistic performance evaluation of newly introduced ...
Abstract. Most research articles presenting new data analysis methods claim that “the new method performs better than existing methods,” but the ...
Read more >
Biology updates - International Baccalaureate®
The new DP biology course will be launched in February 2023 for first teaching in August 2023. First assessment will take place in...
Read more >
How to Change Professional Evaluation in Biology | BioScience
A bold call for a new assessment system for professional productivity in biology appears on p. 619 of this issue.
Read more >
Implementation of a New Quantitative Biology Course
Students currently in the quantitative biology course ... ... Implementation of a New Quantitative Biology Course: Assessment of Students' ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found