question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Mitigate Spyder wiping users' files with more robust atomic saves/autosaves

See original GitHub issue

Problem Description

What steps reproduce the problem?

  1. Save file to disk with connection issues (confirmed)
  2. Crashes during runfile or during saving immediatly prior (reported 2x)
  3. Other problems during saving???

Given we’ve publicly advertised the autosave feature as the solution to the reports we’ve been getting of Spyder completely wiping users’ files on occasion e.g. @ccordoba12 in #6571:

I don’t understand how Spyder can delete your script, but we’re working to provide autosaving functionality for our next major release (Spyder 4).

and particularly now that our own developer @jnsebgosselin is regularly experiencing a such a problem and can reproduce it, we should ensure autosave is reasonably “fail-safer” in the event that saving the file normally fails (silently or otherwise) and wipes or corrupts the file on disk. As currently implemented by @jitseniesen following his hard work in #7660 , the feature is reasonably sophisticated with a nice UI. However, as an initial implementation, it does leave open several significant opportunities for catastrophic data loss in several major scenarios:

Failure mode: Saving may fail silently, or in flushing the data to the storage medium, which would result in the autosave still being deleted and the save being corrupted or wiped (or at least out of date). This would possibly apply to the repeated instances of files being wiped that @jnsebgosselin is experiencing (depending on exactly how the failure occurs).

Mitigations:

  • Hardening the e.g. FileInfo._write_to_file() function, e.g. with <file>.flush() then os.fsync(); switching to the more modern io.open() on Python 2 and making sure files are explicitly closed might help a little too
  • Writing to a tempfile when saving, then doing an atomic replace of the original by the appropriate platform-specific method
  • If there are significant cases where the OS cannot be trusted to throw an error if something goes wring, we might need a readback check for the tempfile vs. the in memory version (at minimum, that the file reads without error and is the correct length, and perhaps comparing the first line and last lines), though this could be user-optional if its not determined to be critical or it takes a non-trivial amount of time
  • Temporarily disable autosave and warn the user if a save failure is detected, and notify the user of this; only re-enable once a manual save is successful

I’ve included the full form of the discussion between various parties on this issue below, for reference in one place.

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:19 (19 by maintainers)

github_iconTop GitHub Comments

1reaction
CAM-Gerlachcommented, Nov 24, 2018

Instead of rolling something on our own to make saves more reliable, we could just use the atomicwrites package. Its already on conda defaults and its what pytest uses.

0reactions
ccordoba12commented, Nov 28, 2018

Thanks!

Read more comments on GitHub >

github_iconTop Results From Across the Web

What IT administrators should know about AutoSave
AutoSave is enabled when a file is stored on OneDrive, OneDrive for work or school, or SharePoint in Microsoft 365. It automatically saves...
Read more >
Why does Spyder want to save my *.py file when I run script?
For some reason Spyder requires that all open files are saved (the tabs on the upper left of the user interface), not just...
Read more >
CHANGELOG.md ... - GitLab
Quickly and easily edit multiple files in your project. ... Edit this file only.
Read more >
Application Data Review - Fedora
DATA OUTPUT COPASI allows users to define report files where data is written ... To make things more interesting, Atomic Tanks also features...
Read more >
Nexus mods and community
We host 412,464 files for 1,913 games from 161,073 authors serving ... We're happy to say we're now serving more users and hosting...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found