Automatically reset buffer when file is read
See original GitHub issueProblem:
read_csv
does not work when multiple files are uploaded
Reason:
It works currently because we are creating a new BytesIO (or StringIO) object each time in deltagenerator. the object gets passed to and gets processed. when another file is uploaded or a rerun happens, we create a new IO object that is starting back at the beginning.
The new implementation does not work because we are using the same buffer and once it’s read the position is not getting reset. need to reset position after read (issues with read_csv 😢)
Several users (here, here, and here) have been stumped by this. We have updated our docs to indicate you should do this.
Issue Analytics
- State:
- Created 3 years ago
- Comments:10 (5 by maintainers)
Top Results From Across the Web
Does Python automatically flush its buffer when calling seek ...
I understand that if I explicitly flush the file with file.flush() or close the file with file.close() then the Python buffer will be ......
Read more >Will buffer be automatically flushed to disk when a process ...
To flush the filesystem buffers, use the sync command. I'd like to read the file right after the command exits, but I do...
Read more >File flush() method in Python - GeeksforGeeks
The flush() method in Python file handling clears the internal buffer of the file. In Python, files are automatically flushed while closing ...
Read more >How do I reload a file in a buffer? - Emacs Stack Exchange
Use `global-auto-revert-mode' to automatically revert all buffers. Use `auto-revert-tail-mode' if you know that the file will only grow without ...
Read more >_flushlbf() — Flush all open line-buffered files - IBM
The system automatically flushes buffers when you close the stream or when a program ends normally without closing the stream. The _flushlbf() function...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Excellent…That solved my problems…onward and upward…
Hi @ikeshare192, sorry to hear about the headache this is causing you. The reason this error is occurring is because of an optimization we made to return the same object on rerun. Previously we were creating a new one on each run which wasn’t ideal. Because we are now returning the same object, the buffer position of the object is persisting.
The solution is to reset the buffer manually like so:
A workaround would be to use
getValue()
instead ofread()
if you are reading the file yourself and not passing this to a library.