Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

document `dvc import` and `dvc get` (previously `pkg` commands)

See original GitHub issue

UPDATE: See description in https://github.com/iterative/dvc/pull/2160 instead.

OLD: Some introductory descriptions for DVC packages (from Discord):

You have project named awesome_project, which has awesome_data in it pkg will allow you to “import” the project dvc pkg import awesome_project://awesome_data (syntax may be incorrect)

Any DVC repo is a package. Outputs inside packages can be called artifacts (that we’ll also be able to work with)

There will be a set of commands to import/export these packages across different repositories and from a repository to a file system ~~(just a data artifact)~~

basic use cases include things like: dataset registry (reuse a single dataset in multiple project, while having a single point of responsibility for the dataset itself), easy mechanism to pull a data artifact w/o doing git clone + dvc pull data.dvc, etc

in the future (no need to focus on this for now) any set of DVC files will be considered as package, thus making pipelines reusable

Issue Analytics

State:
Created 4 years ago
Comments:5 (5 by maintainers)

Top GitHub Comments

1reaction

efiopcommented, Jul 15, 2019

Note: also need to make sure to add a note that old import was moved to import-url.

1reaction

shchekleincommented, Jun 27, 2019

@jorgeorpinel this is a priority 😃 we will see if we merge it before pkg code is merged or not. It will also take some time to get the docs for it done end-to-end. It’s a big feature.

Top Results From Across the Web

import | Data Version Control - DVC

Provides an easy way to reuse files or directories tracked in any DVC repositoryDVC repository (e.g. datasets, intermediate results, ML models) or Git ......

import/get directory with git files and dvc outputs #3087 - GitHub

I just did a: dvc import https://github.com/my_company/my_repo data which gives me a data folder and a data.dvc files in my current folder.

Data & Model Management with DVC | Analytics Vidhya

Search & import files from a DVC repo. Later, we will also look into using DVC as a Python package. Installation. DVC can...

dvc - PyPI

The command should look like this: pip install 'dvc[s3]' (in this case AWS S3 dependencies such as boto3 will be installed automatically). To...

Create a project - Jupyter Tutorial 0.9.0

Before DVC is used, even a remote storage is established. ... You can also import data and models from another project with the...