question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

cached / incremental execution ?

See original GitHub issue

Is your feature request related to a problem? Please describe.

I have the contents of a course that i’m trying to turn into a jupyter book; the initial process to build the book is very tedious, and especially so because

  1. execution always restarts from the beginning
  2. it’s painful to hide a broken notebook

let me explain :

  1. first time I run ‘jupyter book build’ I see the execution of notebooks number, say, 1, 2 and 3; there’s an error in nb3, I fix it; at that point if I re-run ‘jupyter book build’ execution will restart from scratch and I’ll see again notebooks # 1, 2, 3, if I’m lucky 4, and so on; I can’t fathom how many zillion of times I’ll have executed notebook #1 at the end

  2. in order to speed things up I’d like to kind of hide notebooks once they’re ok; so I try to comment them from the _toc file, but apparently that’s not effective; it feels like every single possible notebook in the directory gets executed !

Describe the solution you’d like

It’d make most sense to cache the result of executed notebooks; if that was the case my second point would of course become less of a problem, but I still feel like one should be allowed to have lingering, possibly broken, notebooks around, without that breaking an otherwise perfectly sound book

Additional context

I am wary that I might just have misunderstood, or misconfigured, the tool, I’d love to hear it if that’s the case

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:10 (6 by maintainers)

github_iconTop GitHub Comments

1reaction
parmentelatcommented, Jun 29, 2020

thanks for the answers; indeed exclude_patterns could come in handy !
on the longer run though, it’d be more convenient imho if
a) the tool could consider only the notebooks mentioned in the toc (preferred), or b) at least to have exclusions declared in the same file/location as the toc; (one could even think of having both of them merged, i.e. give the ability to define the toc as a subkey in the config); right now, when excluding one file it’s going to mean repeat the same info twice in 2 separate files…

0reactions
choldgrafcommented, Apr 14, 2021

closing this issue since I think it’s addressed, feel free to re-open if there’s something else to discuss! 🎉

Read more comments on GitHub >

github_iconTop Results From Across the Web

Incremental Mode — Virtual DataPort Administration Guide
When you query a view whose cache mode is incremental, the execution engine executes two queries: A query to the cache database.
Read more >
incremental cache - MSDN - Microsoft
In this article the author introduces incrementally updating the Lookup Cache File by merging the old cache records with the new records then ......
Read more >
Incremental Computation via Function ... - Matthew A. Hammer
about a 50% overhead on the speed of execution in situations where no hits were obtained. In empirical tests of applications such as...
Read more >
Incremental cache files (#4821) · Issues - GitLab.org
Description I've noticed that your cache step basically copies a zip ... UTC 2019: Connecting through ssh@**REMOVED** to execute command.
Read more >
Mastering Gradle Caching and Incremental Builds - Medium
gradle/caches/$GRADLE_VERSION contain jars needed for launching Gradle itself and to start execution of a build. ~/.gradle/caches/transforms-1 ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found