Prose for data

Hey guys, I’ve had a few ideas recently that I’m just now connecting the dots on and wanted to get your thoughts. The premise is using prose.io for data, enabling laypersons to manage basic data without worrying about having a database or what “JSON” is. (ref #772)

As an example, prose’s CSV editor lets users edit CSV files like they would a spreadsheet, and save it to a git repo. This enabled a proof of concept I wrote recently called static-api, which generates a static JSON API from a CSV file. A major limitation with tabular data, of course, is the lack of support for nested data (arrays, objects). So in the case of JKAN, where datasets have many resources, we can’t use the CSV editor.

But what if we could do something with JSON? What if prose knew when you were about to create a “dataset document” and generated a “dataset form” for you to fill in, based on a predefined “dataset” schema? Sounds like the metadata editor, right? So, building on that, what if:

1. Prose’s metadata editor supported JSON Schema? This would allow you to use nested arrays/objects (which I don’t believe is currently possible with prose’s schema support). JSON Schema could certainly be represented as YAML, and you could even use a number of tools to generate the form (I mention a few here).
2. You could specify that prose’s metadata editor is the default view when editing a file in a particular directory. For content-less documents, like in JKAN. If you try opening that repo in prose, you’ll see it’s a bit confusing when presented with these blank markdown files, and you have to know to click the “metadata editor” to get to what you actually want. Perhaps the prose config file could specify to have it shown by default in certain directories.
3. Prose’s metadata editor could generate regular JSON/YAML and not just front-matter. In a sense it would become a data editor, not just a metadata editor. It’s doing most of that now, but it’s putting --- before and after what it generates. I imagine being able to specify in the prose config whether you want it to generate front-matter, regular yaml (no dashes), or json.

I think these 3 suggestions are standalone features themselves which, when combined, could allow laypersons to use prose.io to edit static complex data files in a user-friendly way. Thoughts?

EDIT: Another idea is just to use a JSON editor like this or this and some sort of template/starting document they can clone (perhaps something like github’s issue templates). But in my opinion those editors aren’t very user-friendly.

[timwis]

Thanks @simonv3.

Just thought of another possible path for this (and updated original post): Something closer to github’s issue templates - define a template file with some text in it, and every time you click “new” it prefills it with that text. At the moment, clicking “new” prefills it with:

## A New Post

Enter text in [Markdown](http://daringfireball.net/projects/markdown/). Use the toolbar above, or click the **?** button for formatting help.

It also sets the filename to current-directory/2016-03-08-your-filename.md (note the .md suffix)

What if you could set a template and default file name/extension for each directory? This way you could set a JSON file like:

{
  "foo": "",
  "bar": [
    {"baz": ""}
  ]
}

That, paired with a generic JSON GUID editor like this or this (similar to the CSV editor) could be a starting point.

[simonv3]

Thanks for this writeup @timwis, saw you outline it in #dat - I think this could be really interesting to @bnvk who’s doing some work with data packages in Conjuror to make them auto-magically generated based on certain “spells”).

Also, this is probably of interest to the people at @openfarmcc who will probably settle on a way of storing crop / plant data in JSON/YAML https://github.com/openfarmcc/Crops/issues/6. @andru and @mstenta, I think something like prose.io for editing data would be a great thing for the crop data we’re planning on gathering.

(basically, I’m just pinging all the parties that I think might be interested in this)