[csv-transformer] Make a node from each line

Summary

CSV is very convenient format because it is imitate relational database. Right now (v 0.1.0) csv-transformer makes one node per csv file. And I didn’t find a way to create referencies across different node types because all data stored in records field.

Basic example

It is good to have an option for csv-transformer to make one node per line. One column can be a reference to another node type, e.g. category.

Motivation

It is pretty easy to manage certain types of data in CSV file because you have a bird’s eye view on big data set.

---

This is pretty much a follow up for #293. @hjvedvik mentioned that we need some changes in core to make it possible. Could you enumerate those blockers.

[Al-Rozhkov]

That’s pretty much how I do it.

1. Run yarn add xlsx.
2. Create a folder for custom plugin in the root of your project, e.g. source-xls-plugin.
3. Create index.js inside source-xls-plugin:

const XLSX = require('xlsx')

const createNodes = (contentType, filePath) => {
  const xls = XLSX.readFile(filePath)
  xls.SheetNames.forEach((n) => {
    const parsedContent = XLSX.utils.sheet_to_json(xls.Sheets[n], {raw: false})

    for (let i = 0; i < parsedContent.length; i++) {
      // Check if row contain required columns
      if ('series' in parsedContent[i] && 'name' in parsedContent[i]) {
        contentType.addNode(parsedContent[i])
      }
    }
  })
}

module.exports = function (api) {
  api.loadSource({ addContentType } => {
    
    // Add new content type
    const SomeProductItem = addContentType({
      typeName: 'SomeProductItem'
    })
    SomeProductItem.addReference('series', 'ParentProduct')
    createNodes(SomeProductItem, 'products/path-to-file.xls')
  })
}

1. Register custom plugin inside gridsome.config.js like this:

module.exports = {
  siteName: 'YOUR SITE',
  plugins: [
    {
      use: '~/source-xls-plugin',
      options: {}
    }
  ]
}

[NickStees]

Any update on generating content from CSV files? I really can’t even figure out how to read a single CSV file and get it into GraphQL. I found out its the same as using Markdown source and the transformer auto detects the csv file. It be great to improve the docs for that plugin just a little bit further.