question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Duplicated prediction results when open Auto-Annotation in NER template

See original GitHub issue

Describe the bug A clear and concise description of what the bug is.

I am trying to use label-studio with the feature of Auto-Annotation in a template of NER. But I found that the same prediction from my ML backend will be triggered again and again.

I am not sure if this is the LabelStudio’s responsibility or my ML backend’s responsibility to avoid this behavior.

To Reproduce Steps to reproduce the behavior:

  1. Go to ‘…’
  2. Click on ‘…’
  3. Scroll down to ‘…’
  4. See error

Expected behavior A clear and concise description of what you expected to happen.

I think LabelStudio show also know that the current task is already get annoated by the ML backend so that the same annotation will not be added twice.

Screenshots If applicable, add screenshots to help explain your problem.

Here is a screen record to show the behavior (no voice).

https://user-images.githubusercontent.com/661860/178113109-f4dbae67-e065-4216-9701-d2a0c201188b.mov

In the video you can see no matter how many times I annotated the text, the same prediction result from ML backend will be added to the text.

Environment (please complete the following information):

  • OS: Mac OS 12.4
  • Label Studio Version 1.5.0

Additional context Add any other context about the problem here.

Issue Analytics

  • State:open
  • Created a year ago
  • Comments:6 (6 by maintainers)

github_iconTop GitHub Comments

1reaction
KonstantinKorotaevcommented, Jul 19, 2022

Hi @aisensiy I have created a new feature request for it.

0reactions
aisensiycommented, Jul 18, 2022

Any process about this issue?

Read more comments on GitHub >

github_iconTop Results From Across the Web

Duplicated examples in NER.teach & large jsonl files - usage
Issue 1: I've noticed that we've been seeing lots of duplicates after retraining the model, where we are asked to relabel an example...
Read more >
Label Studio Documentation — Label and annotate data
Start labeling · Choose which tasks to label · Label a region in the data · Label overlapping regions · Select multiple regions...
Read more >
arXiv:2109.03784v1 [cs.LG] 8 Sep 2021
round of active learning, clustering results are tuned based on the prediction results of the current stage. Experimental.
Read more >
Data Labeling: The Authoritative Guide - Scale AI
For example, data labelers will label all cars in a given scene for ... low precision returns many results, but many of the...
Read more >
Problem adding custom entities to SpaCy's NER
In principle the way you're trying to solve the catastrophic forgetting problem, by retraining it on its old predictions, seems like a good ......
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found