Database locked error with SqliteHuey
See original GitHub issueIssue Description
Hi nice library, I like the Sqlite backend so much since it comes pre-installed almost everywhere. My problem is I get a database is locked
error randomly.
[2019-08-22 00:08:50,358] ERROR:huey.consumer.Worker:Worker-2:Error reading from queue
Traceback (most recent call last):
File "/Users/rajaravivarma/.local/share/virtualenvs/youtube-dl-parallel-o-ngZiPY/lib/python3.7/site-packages/huey/consumer.py", line 94, in loop
task = self.huey.dequeue()
File "/Users/rajaravivarma/.local/share/virtualenvs/youtube-dl-parallel-o-ngZiPY/lib/python3.7/site-packages/huey/api.py", line 282, in dequeue
data = self.storage.dequeue()
File "/Users/rajaravivarma/.local/share/virtualenvs/youtube-dl-parallel-o-ngZiPY/lib/python3.7/site-packages/huey/storage.py", line 712, in dequeue
with self.db(commit=True) as curs:
File "/Users/rajaravivarma/.pyenv/versions/3.7.3/lib/python3.7/contextlib.py", line 112, in __enter__
return next(self.gen)
File "/Users/rajaravivarma/.local/share/virtualenvs/youtube-dl-parallel-o-ngZiPY/lib/python3.7/site-packages/huey/storage.py", line 647, in db
if commit: cursor.execute(self.begin_sql)
sqlite3.OperationalError: database is locked
This is the code I am using.
import re
from huey import SqliteHuey
import youtube_dl
huey = SqliteHuey(filename="downloads.db")
@huey.task(retries=2, retry_delay=60)
def download(url):
ignorable_errors = ["HTTP Error 404", "no video in this tweet"]
status = re.match(r"https.*/status/(\d+)", url).groups(0)[0]
ydl_opts = {"quiet": True, "outtmpl": f"{status}.%(ext)s"}
with youtube_dl.YoutubeDL(ydl_opts) as ydl:
try:
ydl.download([url])
except Exception as e:
error_message = str(e)
if any(msg in error_message for msg in ignorable_errors):
pass
else:
raise e
Issue Analytics
- State:
- Created 4 years ago
- Comments:12 (6 by maintainers)
Top Results From Across the Web
Fix SQLite Database File is Locked Error - Kernel Data Recovery
A locked SQLite database stops the user from writing more transactions, and the tables are not updated or altered anymore. If you are...
Read more >How to Fix SQLite “Database Locked” Error - KaplanSoft
This error code occurs when the user tries to perform two inappropriate operations on a database at the same detail and on the...
Read more >Huey's API — huey 2.4.4 documentation - Read the Docs
Raised when Huey encounters a configuration problem. class TaskLockedException ¶. Raised by the consumer when a task lock cannot be acquired. class ...
Read more >huey Documentation - Read the Docs
huey = SqliteHuey(filename='/tmp/demo.db'). @huey.task() ... The consumer puts the error into the result store and the exception is logged.
Read more >database disk image is malformed when using huey task ...
demo.py from huey import SqliteHuey huey = SqliteHuey(filename='/tmp/demo.db') ... 2021-05-20 02:48:29,667] ERROR:huey.consumer.
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Added an option to specify the
timeout
when instantiatingSqliteHuey
.Thanks @coleifer