Standardized description of data imports

It would be worth considering a standardized config file for staging web-accessible data that would. For example data.yml

- src: s3://openneuro.org/ds000113
  dst: /data
  recursive: True

S3, FTP, HTTP and other protocols could be supported. This config file would be read by a script called withing Dockerfile to stage necessary data for the notebook to run. What do people think?

Related to #199.

[chrisgorgo]

Unfortunately, I did not have time to work on this. As a side note, I think that listing different use cases/examples would help scope the feature. One can also try a more data-driven method and use https://www.kaggle.com/github/github-repos/home to see what data intake methods people use in the wild.

On Fri, Feb 15, 2019 at 9:40 AM Chris Holdgraf [e-mail] wrote:

I am afraid I did not get your point, do you mean we need to create a documentation in binder for repo2data or intake ?

Yep, basically I’m just saying “rather than building technical assumptions and dependencies within repo2docker, we’ll make sure to highlight these tools for data import/export/versioning in the documentation and examples we put together”

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jupyter/repo2docker/issues/460#issuecomment-464135956, or mute the thread https://github.com/notifications/unsubscribe-auth/AAOkp6Gx5VAlkJ8H5HEKo9QyuFmehyu0ks5vNvD4gaJpZM4YNGJt .

[choldgraf]

I am afraid I did not get your point, do you mean we need to create a documentation in binder for repo2data or intake ?

Yep, basically I’m just saying “rather than building technical assumptions and dependencies within repo2docker, we’ll make sure to highlight these tools for data import/export/versioning in the documentation and examples we put together”