question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

The cache key for /warm_up_cache/ differs from /explore_json/

See original GitHub issue

Make sure these boxes are checked before submitting your issue - thank you!

  • I have checked the superset logs for python stacktraces and included it here as text if any
  • I have reproduced the issue with at least the latest released version of superset
  • I have checked the issue tracker for the same issue and I haven’t found one similar

Superset version

0.21.0dev

Expected results

Both the /warm_up_cache/ and /explore_json/ API endpoints should produce the same cache key for a given slice. A hashed version of form-data is used for the cache key, though it seems the AJAX form-data is mutated via i) the JavaScript and ii) via the Superset.explore_json(...) method thus the keys are not equivalent.

Actual results

Using the example dataset, the /warm_up_cache/?slice_id=60 endpoint hashes the following string for the cache key:

[
    (u'compare_lag', u'10'),
    (u'compare_suffix', u'o10Y'), 
    (u'granularity', u'ds'), 
    (u'groupby', []), 
    (u'json', u'false'), 
    (u'limit', u'100'), 
    (u'markup_type', u'markdown'), 
    (u'metric', u'sum__num'), 
    (u'metrics', [u'sum__num']), 
    (u'rotation', u'square'), 
    (u'row_limit', 50000), 
    (u'series', u'name'), 
    (u'since', u'100 years ago'),
    (u'size_from', u'10'),
    (u'size_to', u'70'), 
    (u'slice_id', 60), 
    (u'slice_name', u'Name Cloud'), 
    (u'until', u'now'), 
    (u'viz_type', u'word_cloud'), 
    (u'where', u'')
]

whereas /slice/60/ endpoint hashes the following string for the cache key:

[
    (u'color_scheme', u'bnbColors'), 
    (u'datasource', u'3__table'), 
    (u'filters', []), 
    (u'granularity_sqla', u'ds'), 
    (u'having', u''), 
    (u'limit', u'100'), 
    (u'metric', u'sum__num'), 
    (u'rotation', u'square'),
    (u'series', u'name'), 
    (u'since', u'100 years ago'), 
    (u'size_from', u'10'), 
    (u'size_to', u'70'), 
    (u'slice_id', 60), 
    (u'time_grain_sqla', u'Time Column'), 
    (u'until', u'now'), 
    (u'viz_type', u'word_cloud'), 
    (u'where', u'')
]

I’m not sure what the correct way to proceed is. I presume there are a couple of options:

  1. Update the cache key logic to contain a whitelist of form-data fields to hash. Note this may be difficult to maintain. In essence differentiate between form-data elements which are related to the payload and those which are not.
  2. Deprecate the /warm_up_cache/ API endpoint and augment existing non-API endpoints to pass the force request argument, i.e., /slice/60/?force=true. Note the issue with this approach is one may need to use a headless browser if the intent is to systematically warm up the cache.

Steps to reproduce

See above.

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Comments:6 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
ghulandscommented, Apr 24, 2018

@john-bodley can this be closed?

0reactions
ukm21commented, Jul 7, 2021

facing this issue in superset 1.1.0 where cache key is different for warm_up_cache and explore_chart

Read more comments on GitHub >

github_iconTop Results From Across the Web

The cache key for /warm_up_cache/ differs from /explore_json
Both the /warm_up_cache/ and /explore_json/ API endpoints should produce the same cache key for a given slice. A hashed version of form-data is ......
Read more >
Cache warming with RABL for JSON templates - Stack Overflow
When calling Rabl.render vs API calls, the caches generated do not have the same cache-keys. When using Rabl.render directly should I expect ...
Read more >
Caching Dependencies - CircleCI
This document is a guide to caching dependencies in CircleCI pipelines. ... keys: # Find a cache corresponding to this specific package-lock.json checksum ......
Read more >
Performance and Debugging — Panel v0.14.2rc2
The as_cached · The first time the app is loaded the data will be cached and subsequent sessions will simply look up the...
Read more >
Time Series Caching with Python and Redis - Roman Imankulov
Overall, the task was relatively straightforward: we had to call TransactionFetcher. fetch() for hot keys. To schedule cache warmup jobs, we ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found