question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[FEATURE] injest fiftyone datasets

See original GitHub issue

🚨🚨 Feature Request

  • Related to an existing Issue
  • A new implementation (Improvement, Extension)

Is your feature request related to a problem?

My problem is being able to ingest fiftyone datasets into deeplake

  • exporting would also be an interesting addition

If your feature will improve HUB

Fiftyone is a common dataset import and export tool, integration with deeplake would make such operations easy, and would mean that we do not have to implement such operations from scratch.

Description of the possible solution

import deeplake
import fiftyone

# ideally this would be able to detect the various different types and labels and be able to import these accordingly.
dataset = fiftyone.load_dataset('my_dataset')
deeplake.ingest_51('deeplake_data/my_dataset', dataset)

An alternative solution to the problem can look like

Ingest steps could be written manually. (Fiftyone doesn’t enforce much structure on the datasets so I am not sure if the original ingest function even has a distinct solution, maybe some basic structure would be required).

Teachability, Documentation, Adoption, Migration Strategy Needs discussion first

Issue Analytics

  • State:open
  • Created a year ago
  • Comments:7 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
istraniccommented, Oct 24, 2022

Got it. Thanks @nmichlo!

0reactions
nmichlocommented, Oct 24, 2022

@istranic no problem, glad to help!

Ideally in the long run I personally would prefer not to use fiftyone, and ingest/export datasets directly.

However, I think there might be merit for both?

  • ingesting directly from fiftyone might keep additional information that would otherwise be discarded if there is first an export and then import step.
  • this would also allow easier migration to deeplake
Read more comments on GitHub >

github_iconTop Results From Across the Web

Using FiftyOne Datasets - Voxel51
After a Dataset has been loaded or created, FiftyOne provides powerful functionality to inspect, search, and modify it from a Dataset -wide down...
Read more >
fiftyone.core.dataset - Voxel51
FiftyOne datasets ingest and store the labels for all samples internally; raw media is stored on disk and the dataset provides paths to...
Read more >
Creating Views — FiftyOne 0.18.0 documentation - Voxel51
Dataset views and the view expressions language are powerful and flexible aspects of FiftyOne. Getting comfortable with using views and expressions to slice...
Read more >
FiftyOne Basics - Voxel51
FiftyOne Datasets allow you to easily load, modify, visualize, and evaluate your data along with any related labels (classifications, detections, etc).
Read more >
Using Sample Parsers — FiftyOne 0.18.0 documentation
The SampleParser interface provides native support for loading samples in a variety of common formats, and it can be easily extended to import...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found