question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Support importing datasets in JSON format

See original GitHub issue

Currently in order to upload a dataset for annotation all of the documents in the dataset must be in a single txt file with each document separated by a newline character like so:

document 1 text
document 2 text

It would be very helpful if instead documents could be within separate files and a bulk upload could be performed. For example:

Document 1

Document
1
text

Document 2

Document
2
text

This would also allow documents to contain newline characters which was requested here: #12

Love the tool!

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Reactions:2
  • Comments:14 (6 by maintainers)

github_iconTop GitHub Comments

1reaction
Zerthickcommented, Oct 23, 2018

I agree, uploading a tar.gz archive would be helpful as well.

1reaction
Zerthickcommented, Oct 16, 2018

Alternatively, supporting uploading a single file in JSON format such as:

[
    "Document 1 text",
    "Document 2 text"
]

Would also work.

Edit: For example this snippet of code would parse the above JSON:

form_data = json.load(request.FILES['json_file'].file)
    Document.objects.bulk_create(
        [Document(text=entry, project=project) for entry in form_data])

There appears to be a gotcha in your rendering code though where even if the document text contains a newline character it is not rendered as such. Haven’t figured out where that occurs yet.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Import data from JSON - Atlassian Support
The JSON import feature allows you to import issues from external issue trackers that cannot be exported to CSV files. Please note that...
Read more >
Data I/O: JSON Import and Export - RAI Documentation
This how-to guide demonstrates how to import and export JSON data using Rel.
Read more >
How to Import data into Directus using JSON
Open the collection where you intend to add the data. Expand the right menu and click on the Import/Export section. Click on the...
Read more >
Importing data from JSON files and Power BI Rest APIs into ...
JSON stands for JavaScript Object Notation and is one of the most commonly used formats for data exchange between different platforms and ...
Read more >
Working with JSON | Trifacta Help Center
Import the file · You can import your JSON file using the Import data page. Navigate and select your file from the source...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found