question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Locally caching remote or inline resources via "cache" key

See original GitHub issue

As of 1.0.0-beta.15, only one of path, url, or data is allowed on a given resource. In discussing this, it was suggested that it is useful to have a local path to which to download a remote (url) or inline (data) resource so that non-datapackage-aware tooling can access. As a result of discussion in #223, it has been suggested that a new key, cache, be used on a resource to specify a local path to “cache” remote or inline resources to disk.

Examples:

{
  "url": "http://xyz.com/dataset.csv",
  "cache": "dataset.csv"
}
{
  "data": [
    {"a": "1", "b": "2"}
  ],
  "cache": "dataset.csv"
}

So, when a resource has a url or data key, it SHOULD also have a cache key. An implementation that is reading the data in the data package, should then first check cache for the data, if it exists, and, if not, save the data as specified in url or data to cache. Otherwise, cache acts just like path. When a resource has a path key, an implementation should ignore cache.

WDYT? @morty @amercader @rgrp @pwalsh

Issue Analytics

  • State:closed
  • Created 8 years ago
  • Comments:25 (21 by maintainers)

github_iconTop GitHub Comments

1reaction
rufuspollockcommented, Nov 17, 2016

Removing from v1 milestone as now a pattern item.

0reactions
pwalshcommented, Dec 19, 2016

This was done in #331 and will close when it is merged.

Read more comments on GitHub >

github_iconTop Results From Across the Web

CACHE command to support shared remote caches #1540
Currently the CACHE command only works with local cache, however, supporting remote cache would allow multiple users to share the same cache ...
Read more >
Remote caching - Earthly
This page only covers remote shared caching through the use of a registry. ... In CI, read-only inline cache (typically in PR builds):....
Read more >
Configuring Infinispan caches
Starting Infinispan Server automatically instantiates the Cache Manager so you can create and delete remote caches and other resources like ...
Read more >
Enabling Docker Layer Caching - CircleCI
Docker layer caching can be used with both the machine executor and in the remote Docker environment ( setup_remote_docker ).
Read more >
Client-side caching in Redis
The Redis implementation of client-side caching · Clients can enable tracking if they want. · When tracking is enabled, the server remembers what...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found