Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Predictions for certain paragraphs are inaccurate. What's wrong?

See original GitHub issue

I have been testing cdQA on paragraphs that have been generated from CSV. I convert the structured data into English, then predict answers using BERT.

I’ve described the approach here:

I combine 2 or 3 sentences into paragraphs, then concatenate multiple paragraphs into one dataframe for cdQA pipeline, then query the dataset but results are often incorrect. An example of a sentence:

According to our website, the Melbourne Convention Centre & South Wharf Precinct
project is located at 1 Convention Centre Pl, South Wharf VIC 3006, Australia. The
Melbourne Convention Centre & South Wharf Precinct project has won three awards. The project started in 2014 and was completed in 2016.

And query

how many awards has the Melbourne Convention Centre project won?

Could this form of English writing be too dissimilar to the corpora and datasets on which BERT was pre-trained and fine-tuned? Can you suggest how I could improve results? Thanks.

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:25

github_iconTop GitHub Comments

radsimucommented, Sep 24, 2019

finally I found the problem: says cdqa_pipeline = QAPipeline(model='bert_qa_vCPU-sklearn.joblib')

should be cdqa_pipeline = QAPipeline(reader='bert_qa_vCPU-sklearn.joblib')

JimAvacommented, Sep 19, 2019

Hi @andrelmfarias ,

I’ve been trying out the newly introduced retrieve based on BM25 and it seems to be working great! Thank you for your efforts. Will provide more updates…

Read more comments on GitHub >

github_iconTop Results From Across the Web

When wrong predictions provide more support than right ones
Imagine a new geophysical theory that leads to predictions as to when large earthquakes, which are rare, will occur. Which outcome would leave...
Read more >
All Forecasts Are Wrong - LinkedIn
How then to maximize the accuracy of your forecasting? Here are some useful tips: Know and embrace your errors. No one is Nostradamus....
Read more >
What Research Tells Us About Making Accurate Predictions
“Prediction is very difficult,” the old chestnut goes, “especially about the future.” And for years, social science agreed.
Read more >
How To Teach Students to Make Predictions While They Read
Be sure to choose a book that provide obvious opportunities for students make predictions. This will help set students up for success and...
Read more >
The accuracy, fairness, and limits of predicting recidivism
We have shown that commercial software that is widely used to predict recidivism is no more accurate or fair than the predictions of...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found