Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Repeating OperationalError: (sqlite3.OperationalError) database is locked

See original GitHub issue

I’m getting these errors, while my box upgrades itself at 4 AM using unattended-upgrades. There are some related tickets: #703, #336 and Trac issue #3030 - finally commit 0c74a6b

2016-03-29 04:00 ERROR    lazy_lookup   nC_movie-HUN    Unhandled error in lazy lookup plugin
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/flexget/utils/lazy_dict.py", line 36, in __getitem__
    func(self.store)
  File "/usr/local/lib/python2.7/dist-packages/flexget/plugins/metainfo/imdb_lookup.py", line 235, in lazy_loader
    self.lookup(entry)
  File "/usr/local/lib/python2.7/dist-packages/flexget/plugin.py", line 124, in wrapped_func
    return func(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/flexget/utils/database.py", line 30, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/flexget/plugins/metainfo/imdb_lookup.py", line 356, in lookup
    entry.update_using_map(self.field_map, movie)
  File "/usr/local/lib/python2.7/dist-packages/flexget/entry.py", line 266, in update_using_map
    v = value(source_item)
  File "/usr/local/lib/python2.7/dist-packages/flexget/plugins/metainfo/imdb_lookup.py", line 212, in <lambda>
    'imdb_languages': lambda movie: [lang.language.name for lang in movie.languages],
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/attributes.py", line 237, in __get__
    return self.impl.get(instance_state(instance), dict_)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/attributes.py", line 583, in get
    value = self.callable_(state, passive)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/strategies.py", line 532, in _load_for_state
    return self._emit_lazyload(session, state, ident_key, passive)
  File "<string>", line 1, in <lambda>
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/strategies.py", line 602, in _emit_lazyload
    result = q.all()
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2588, in all
    return list(self)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2736, in __iter__
    return self._execute_and_instances(context)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2751, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 914, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
    context)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1341, in _handle_dbapi_exception
    exc_info
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 200, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
    context)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 450, in do_execute
    cursor.execute(statement, parameters)
OperationalError: (sqlite3.OperationalError) database is locked [SQL: u'SELECT imdb_movie_languages.movie_id AS imdb_movie_languages_movie_id, imdb_movie_languages.language_id AS imdb_movie_languages_language_id, imdb_movie_languages.prominence AS imdb_movie_languages_prominence \nFROM imdb_movie_languages \nWHERE ? = imdb_movie_languages.movie_id ORDER BY imdb_movie_languages.prominence'] [parameters: (1495,)]

2016-03-31 04:05 CRITICAL task_queue                    Database error while running a task. Attempting to recover.
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/flexget/task_queue.py", line 47, in run
    self.current_task.execute()
  File "/usr/local/lib/python2.7/dist-packages/flexget/task.py", line 69, in wrapper
    return func(self, *args, **kw)
  File "/usr/local/lib/python2.7/dist-packages/flexget/task.py", line 579, in execute
    self._execute()
  File "/usr/local/lib/python2.7/dist-packages/flexget/task.py", line 511, in _execute
    last_hash = session.query(TaskConfigHash).filter(TaskConfigHash.task == self.name).first()
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2634, in first
    ret = list(self[0:1])
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2457, in __getitem__
    return list(res)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2736, in __iter__
    return self._execute_and_instances(context)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2751, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 914, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
    context)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1341, in _handle_dbapi_exception
    exc_info
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 200, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
    context)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 450, in do_execute
    cursor.execute(statement, parameters)
OperationalError: (sqlite3.OperationalError) database is locked [SQL: u'SELECT feed_config_hash.name AS feed_config_hash_name, feed_config_hash.id AS feed_config_hash_id, feed_config_hash.hash AS feed_config_hash_hash \nFROM feed_config_hash \nWHERE feed_config_hash.name = ?\n LIMIT ? OFFSET ?'] [parameters: (u'nC_notify-Progs', 1, 0)]

2016-04-01 03:57 CRITICAL task_queue                    Database error while running a task. Attempting to recover.
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/flexget/task_queue.py", line 47, in run
    self.current_task.execute()
  File "/usr/local/lib/python2.7/dist-packages/flexget/task.py", line 69, in wrapper
    return func(self, *args, **kw)
  File "/usr/local/lib/python2.7/dist-packages/flexget/task.py", line 579, in execute
    self._execute()
  File "/usr/local/lib/python2.7/dist-packages/flexget/task.py", line 511, in _execute
    last_hash = session.query(TaskConfigHash).filter(TaskConfigHash.task == self.name).first()
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2634, in first
    ret = list(self[0:1])
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2457, in __getitem__
    return list(res)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2736, in __iter__
    return self._execute_and_instances(context)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2751, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 914, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
    context)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1341, in _handle_dbapi_exception
    exc_info
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 200, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
    context)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 450, in do_execute
    cursor.execute(statement, parameters)
OperationalError: (sqlite3.OperationalError) database is locked [SQL: u'SELECT feed_config_hash.name AS feed_config_hash_name, feed_config_hash.id AS feed_config_hash_id, feed_config_hash.hash AS feed_config_hash_hash \nFROM feed_config_hash \nWHERE feed_config_hash.name = ?\n LIMIT ? OFFSET ?'] [parameters: (u'nC_notify-Progs', 1, 0)]

2016-04-02 12:43 CRITICAL task          nC_notify-Progs BUG: Unhandled error in plugin remember_rejected: (sqlite3.OperationalError) database is locked [SQL: u'DELETE FROM remember_rejected_entry WHERE remember_rejected_entry.feed_id = ? AND remember_rejected_entry.expires < ?'] [parameters: (2109, '2016-04-02 12:43:28.181146')]
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/flexget/task.py", line 438, in __run_plugin
    return method(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/flexget/event.py", line 22, in __call__
    return self.func(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/flexget/plugins/filter/remember_rejected.py", line 94, in on_task_start
    filter(RememberEntry.expires < datetime.now()).delete()
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 3048, in delete
    delete_op.exec_()
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 1127, in exec_
    self._do_exec()
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 1311, in _do_exec
    mapper=self.mapper)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1034, in execute
    bind, close_with_result=True).execute(clause, params or {})
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 914, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
    context)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1341, in _handle_dbapi_exception
    exc_info
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 200, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
    context)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 450, in do_execute
    cursor.execute(statement, parameters)
OperationalError: (sqlite3.OperationalError) database is locked [SQL: u'DELETE FROM remember_rejected_entry WHERE remember_rejected_entry.feed_id = ? AND remember_rejected_entry.expires < ?'] [parameters: (2109, '2016-04-02 12:43:28.181146')]

1000th issue 😃 💯

Issue Analytics

  • State:closed
  • Created 7 years ago
  • Comments:66 (48 by maintainers)

github_iconTop GitHub Comments

2reactions
cviumcommented, Jun 7, 2018

@frankyw @zosky Good news and bad news.

Good news: I’ve located the issue. The problem is with lazy fields (eg. trakt_lookup, imdb_lookup) and backlog plugin, which is used internally by series timeframe. More specifically, this line https://github.com/Flexget/Flexget/blob/develop/flexget/plugins/filter/series.py#L2131 forces an evaluation of all lazy fields, which runs in a separate db session, but series already has a lock on the db at this point, which is why it crashes.

A quick fix is to add object_session(episode).commit() before this line or simply don’t use lookup plugins with timeframe.

Bad news: It’s not an easy thing to fix. It’s a fundamental problem with how we’ve implemented lazy lookups. It could take a long while to fix this properly. Sorry.

1reaction
zoskycommented, May 17, 2018

disabled the cron job and restarted daemon with debug – will have a look in the morning

Read more comments on GitHub >

github_iconTop Results From Across the Web

OperationalError: database is locked - python - Stack Overflow
OperationalError : database is locked errors indicate that your application is experiencing more concurrency than sqlite can handle in default ...
Read more >
Resolve Error and Unlock SQLite Database - ConverterTools
This is an Operational error which indicates that the application is handling more concurrency than the default configuration. it means that one thread...
Read more >
OperationalError: database is locked - Intellipaat Community
The practical reason for this is usually that the python or django shells have opened a request to the DB and it has...
Read more >
Fix SQLite Database File is Locked Error - Kernel Data Recovery
Normally, the error occurs when two users try to run transactions on the same tables and change the content. SQLite engine finds it...
Read more >
File Locking And Concurrency In SQLite Version 3
Open the database file and obtain a SHARED lock. · Check to see if the database file has a hot journal. · Acquire...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

Identified_by doesn’t recognise dates correctly

Next page

Bug since version 1.2.484