[Utilities] Create caching utility

We need a helper function that will get/set/delete files from a cache directory.

On a local machine that would be the .netlify/cache dir

Inside Netlify CI that directory is path.join('/opt/build/cache')

The caching utilities should be smart and save file hashes. A v1 attempt at this was done over here: https://github.com/DavidWells/cache-me-outside

These utilities will be used by plugin authors to make it easier for them to interact with the file cache.

File cache might be used for file storage or for storing values between builds. E.g. the google lighthouse plugin storing page scores between builds

[DavidWells]

Thinking something like this. save (or set), get, delete (or remove)

// Simple save values to cache
const cacheData = cache.save(path, contents)

// Save values to cache
const cacheData = cache.save(path, contents, {
  // time to live
  expires: '14 days'
})

// Simple get cache, no checks for invalid or not
const filePaths = cache.get(path)

// Get cached file or directory
const filePaths = cache.get(path, {
  shouldCacheUpdate: (cacheData) => {
    /* any kind of logic here
      - check date cache last updated
      - Make remote api request and diff a value
      - Diff dependencies files like package.json
      - Whatever you want
    */

    // Return true if cache needs to be updated. False if cache is good to use
    return true
  },
  handleCacheUpdate: () => {
    // exec command / custom logic to refresh the cache
  }
})

// Delete
cache.delete(path)

The cacheData being referred to above is something like:

{
  "createdOn": 1534296638475,
  "createdOnDate": "Wed, 15 Aug 2018 01:30:38 GMT",
  "modifiedOn": 1534300695541,
  "cacheDir": "/Users/davidwells/Netlify/cache-magic/cache/Netlify/cache-magic/serverless-test",
  "cacheDirContents": "/Users/davidwells/Netlify/cache-magic/cache/Netlify/cache-magic/serverless-test/.serverless",
  "contents": {
    "src": "/Users/davidwells/Netlify/cache-magic/serverless-test/.serverless",
    "hash": "0496d16c0a8b1d43ca2d3c77ca48a8e237fdb625",
    "files": {
      "stuff.txt": "11b80f260a5eea9e867a23ab7f96aff77080ff90"
    }
  }
}

This information is available for the shouldCacheUpdate function to quickly diff hashes. That data is also returned from the save method

See how should update works in this flow chart.

I wasn’t too crazy about cache-me-outside’s API taking an array of these config objects. The updating (shouldUpdate + handleUpdate) logic works really well though.

There is also a diff util that is available in here. This is very handy for not reimplementing hashing logic.

Example:

cache.get('path', {
  handleCacheUpdate: 'npm install && echo "this runs when cache is invalid"',
  shouldCacheUpdate: async (cacheManifest, utils) => {
    const packageJson = path.join(__dirname, 'package.json')
    const packageJsonChanged = await utils.diff(packageJson)
    return packageJsonChanged
  }
})

[ehmicky]

README improved in #704.