original errors are swallowed by non-retriable error during savepoint release
See original GitHub issueUPDATE: simplified the gist to only use sanctioned cockroachdb-python code in the simplest ‘create duplicate’ scenario.
The following code returns sqlalchemy.exc.InternalError
because a nested transaction was rolled back. In such a case, I’d like to handle the actual error (eg., a sqlalchemy.IntegrityError
due to duplicate insertion) instead.
You can reproduce this with the following code: https://gist.github.com/gpaul/ead175558dddad2b0ea6479f2405bc8c
That yields the following stacktrace showing the actual reason for the failure: sqlalchemy.IntegrityError
raised the original exception:
Traceback (most recent call last):
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
context)
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
cursor.execute(statement, parameters)
psycopg2.IntegrityError: duplicate key value (key)=('some-key') violates unique constraint "configs_key_key"
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 2223, in _flush
flush_context.execute()
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/orm/unitofwork.py", line 389, in execute
rec.execute(self)
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/orm/unitofwork.py", line 548, in execute
uow
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/orm/persistence.py", line 181, in save_obj
mapper, table, insert)
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/orm/persistence.py", line 835, in _emit_insert_statements
execute(statement, params)
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 945, in execute
return meth(self, multiparams, params)
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
compiled_sql, distilled_params
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
context)
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1402, in _handle_dbapi_exception
exc_info
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 186, in reraise
raise value.with_traceback(tb)
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
context)
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.IntegrityError: (psycopg2.IntegrityError) duplicate key value (key)=('some-key') violates unique constraint "configs_key_key"
[SQL: 'INSERT INTO configs (key, value) VALUES (%(key)s, %(value)s) RETURNING configs.id'] [parameters: {'key': 'some-key', 'value': 'some-value'}]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
context)
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
cursor.execute(statement, parameters)
psycopg2.InternalError: SAVEPOINT COCKROACH_RESTART has not been used or a non-retriable error was encountered: current transaction is aborted, commands ignored until end of transaction block
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "repro4.py", line 66, in <module>
run_transaction(_session_factory, callback)
File "repro4.py", line 30, in run_transaction
return ret
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 567, in __exit__
self.rollback()
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 187, in reraise
raise value
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 564, in __exit__
self.commit()
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 461, in commit
self._prepare_impl()
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 441, in _prepare_impl
self.session.flush()
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 2139, in flush
self._flush(objects)
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 2259, in _flush
transaction.rollback(_capture_exception=True)
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/util/langhelpers.py", line 76, in __exit__
compat.reraise(type_, value, traceback)
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 187, in reraise
raise value
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 2259, in _flush
transaction.rollback(_capture_exception=True)
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 518, in rollback
util.reraise(*rollback_err)
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 187, in reraise
raise value
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 490, in rollback
transaction._rollback_impl()
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 527, in _rollback_impl
t[1].rollback()
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1621, in rollback
self._do_rollback()
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1683, in _do_rollback
self._savepoint, self._parent)
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 750, in _rollback_to_savepoint_impl
self.engine.dialect.do_rollback_to_savepoint(self, name)
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/cockroachdb/sqlalchemy/dialect.py", line 156, in do_rollback_to_savepoint
connection.execute('ROLLBACK TO SAVEPOINT cockroach_restart')
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 939, in execute
return self._execute_text(object, multiparams, params)
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1097, in _execute_text
statement, parameters
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
context)
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1402, in _handle_dbapi_exception
exc_info
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 186, in reraise
raise value.with_traceback(tb)
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
context)
File "/home/gustav/.pyenv/versions/bouncer/lib/python3.5/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.InternalError: (psycopg2.InternalError) SAVEPOINT COCKROACH_RESTART has not been used or a non-retriable error was encountered: current transaction is aborted, commands ignored until end of transaction block
[SQL: 'ROLLBACK TO SAVEPOINT cockroach_restart']
Issue Analytics
- State:
- Created 6 years ago
- Comments:13 (6 by maintainers)
Top Results From Across the Web
original errors are swallowed by non-retriable error during ...
The nested transaction scheme was chosen for compatibility with the sqlalchemy ORM: it knows about rolling back to a savepoint and can restore ......
Read more >Autorollback in postgres using PDO - php - Stack Overflow
If you wanted to be able to swallow exceptions and keep on using the transaction, you must create a SAVEPOINT before each statement...
Read more >cockroachdb digest since Thu Jan 2 02:30:00 UTC 2020
When running with the rand-tables configuration, there's a 50% chance of all tables but the first one to get interleaved into a random...
Read more >Error and Transaction Handling in SQL Server Part Two
My theory is that internal errors in SQL Server result in level 20, while an exception in SSMS/SqlClient only produces a level 11...
Read more >yaojingguo/cockroach:executor.go - VeryToolz
A parse error occurred: we can't determine if there were multiple ... "SAVEPOINT %s has not been used or a non-retriable error was...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
I think cockroachdb/cockroach#16846 solves the main problem here, which is that after certain errors the
ROLLBACK TO SAVEPOINT
would reliably fail.There’s a secondary issue that if
ROLLBACK TO SAVEPOINT
fails for other reasons (e.g. network failure), its error clobbers the underlying error that caused the rollback. I don’t think we need to do anything special here - Python 3’s exception chaining already provides all the errors, so nothing is completely hidden, and there’s no special handling that is necessary when these network errors occur (unlike with the original error here, in which the ConditionFailedError might be handled specially).Thank you, this was fixed in https://github.com/cockroachdb/cockroach/pull/16846