Change SECRET_KEY and get error `ValueError: Invalid decryption key` on K8s

When I try change SECRET_KEY = '\2\1thisismyscretkey\1\2\e\y\y\h' (default), to (for example):

SECRET_KEY = '\2\1888D237694D70843CDBC38195215E5623EDAA74C3D47E8C4F1156\1\2\e\y\y\h'

I getting the error ValueError: Invalid decryption key in sqlalchemy_utils when on execute superset-init.

To reproduce:

$ cd ./install/helm/superset 
$ mkdir config/ && touch config/superset_config.py
$ nano config/superset_config.py
---
import os

# Your App secret key
SECRET_KEY = '\2\1<new long random string>\1\2\e\y\y\h'

# The SQLAlchemy connection string to your database backend
# This connection defines the path to the database that stores your
# superset metadata (slices, connections, tables, dashboards, ...).
# Note that the connection information to connect to the datasources
# you want to explore are managed directly in the web UI
# SQLALCHEMY_DATABASE_URI = 'sqlite:////var/lib/superset/superset.db'
SQLALCHEMY_DATABASE_URI = 'postgresql+psycopg2://XXXXX:XXXX@host/db'

# Flask-WTF flag for CSRF
WTF_CSRF_ENABLED = True
# Add endpoints that need to be exempt from CSRF protection
WTF_CSRF_EXEMPT_LIST = []

# Set this API key to enable Mapbox visualizations
MAPBOX_API_KEY = os.environ.get("MAPBOX_API_KEY", "")
---
$ helm upgrade --install superset .
$ kubectl exec -it superset-<pod_id> superset-init

Expected results

Actual results

Loaded your LOCAL configuration at [/etc/superset/superset_config.py]
/usr/local/lib/python3.6/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
  """)
2019-11-10 14:44:31,592:INFO:root:Configured event logger of type <class 'superset.utils.log.DBEventLogger'>
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
Loaded your LOCAL configuration at [/etc/superset/superset_config.py]
/usr/local/lib/python3.6/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
  """)
2019-11-10 14:44:38,625:INFO:root:Configured event logger of type <class 'superset.utils.log.DBEventLogger'>
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 124, in decrypt
    decrypted = decrypted.decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xca in position 1: invalid continuation byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/superset", line 31, in <module>
    cli()
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/flask/cli.py", line 586, in main
    return super(FlaskGroup, self).main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/flask/cli.py", line 426, in decorator
    return __ctx.invoke(f, *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/superset/cli.py", line 51, in init
    utils.get_or_create_main_db()
  File "/usr/local/lib/python3.6/site-packages/superset/utils/core.py", line 948, in get_or_create_main_db
    get_main_database()
  File "/usr/local/lib/python3.6/site-packages/superset/utils/core.py", line 971, in get_main_database
    return get_or_create_db("main", conf.get("SQLALCHEMY_DATABASE_URI"))
  File "/usr/local/lib/python3.6/site-packages/superset/utils/core.py", line 956, in get_or_create_db
    db.session.query(models.Database).filter_by(database_name=database_name).first()
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3228, in first
    ret = list(self[0:1])
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3018, in __getitem__
    return list(res)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 105, in instances
    util.raise_from_cause(err)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 153, in reraise
    raise value
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 85, in instances
    rows = [proc(row) for row in fetch]
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 85, in <listcomp>
    rows = [proc(row) for row in fetch]
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 572, in _instance
    populators,
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 693, in _populate_full
    dict_[key] = getter(row)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/type_api.py", line 1247, in process
    return process_value(impl_processor(value), dialect)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 409, in process_result_value
    decrypted_value = self.engine.decrypt(value)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 126, in decrypt
    raise ValueError('Invalid decryption key')
ValueError: Invalid decryption key

Screenshots

If applicable, add screenshots to help explain your problem.

How to reproduce the bug

1. Go to ‘…’
2. Click on ‘…’
3. Scroll down to ‘…’
4. See error

Environment

(please complete the following information):

• superset version: superset version 0.34.1
• python version: python --version 3.6.1
• node.js version: node -v
• npm version: npm -v

Checklist

Make sure these boxes are checked before submitting your issue - thank you!

• I have checked the superset logs for python stacktraces and included it here as text if there are any.
• I have reproduced the issue with at least the latest released version of superset.
• I have checked the issue tracker for the same issue and I haven’t found one similar.

Additional context

Add any other context about the problem here.

[jhult]

The databases page would not load for me after changing the SECRET_KEY. Here is how I fixed it:

psql -d superset -U superset -W
# Password for user superset:
select database_name, password from dbs;
update dbs set password = null;

Now the databases page will load but clicking edit on certain databases still didn’t work and caused this stack trace:

Unable to load dialect <class 'sqlalchemy.dialects.mssql.adodbapi.MSDialect_adodbapi'>: type object 'MSDialect_adodbapi' has no attribute 'dbapi'
2021-09-17 18:29:43,812:WARNING:superset.db_engine_specs:Unable to load dialect <class 'sqlalchemy.dialects.mssql.adodbapi.MSDialect_adodbapi'>: type object 'MSDialect_adodbapi' has no attribute 'dbapi'
2021-09-17 18:29:43,819:ERROR:root:Invalid decryption key
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 126, in decrypt
    decrypted = decrypted.decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xae in position 2: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/flask_appbuilder/api/__init__.py", line 85, in wraps
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/flask_appbuilder/api/__init__.py", line 155, in wraps
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/flask_appbuilder/api/__init__.py", line 1448, in get
    return self.get_headless(pk, **kwargs)
  File "/app/superset/utils/log.py", line 242, in wrapper
    value = f(*args, **kwargs)
  File "/app/superset/views/base_api.py", line 390, in get_headless
    duration, response = time_function(super().get_headless, pk, **kwargs)
  File "/app/superset/utils/core.py", line 1461, in time_function
    response = func(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/flask_appbuilder/api/__init__.py", line 1344, in get_headless
    item = self.datamodel.get(pk, self._base_filters, self.show_select_columns)
  File "/usr/local/lib/python3.7/site-packages/flask_appbuilder/models/sqla/interface.py", line 935, in get
    query, _filters, select_columns=select_columns
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3459, in one_or_none
    ret = list(self)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 100, in instances
    cursor.close()
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
    with_traceback=exc_tb,
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
    raise exception
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 80, in instances
    rows = [proc(row) for row in fetch]
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 80, in <listcomp>
    rows = [proc(row) for row in fetch]
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 588, in _instance
    populators,
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 725, in _populate_full
    dict_[key] = getter(row)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/sql/type_api.py", line 1278, in process
    return process_value(impl_processor(value), dialect)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 469, in process_result_value
    value = super().process_result_value(value=value, dialect=dialect)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 414, in process_result_value
    decrypted_value = self.engine.decrypt(value)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 128, in decrypt
    raise ValueError('Invalid decryption key')
ValueError: Invalid decryption key

Create a new database (e.g. name = “New DB”) from the UI. You can get the SQLAlchemy URI using this SQL:

select sqlalchemy_uri from dbs where database_name = 'Original DB';

Once the new database is created, back in psql:

select database_name, id from dbs where database_name = 'New DB' or database_name = 'Original DB';
# 5 = New DB
# 2 = Original DB
update tables set database_id = 5 where database_id = 2;
update tab_state set database_id = 5 where database_id = 2;
update query set database_id = 5 where database_id = 2;
update saved_query set db_id = 5 where db_id = 2;
update table_schema set database_id = 5 where database_id = 2;
delete from dbs where database_name = 'Original DB';
update dbs set database_name = 'Original DB' where database_name = 'New DB';

[dpgaspar]

Hi @newtonjose,

This is probably because you already have database connections that were encrypted with the old key. You have to recreate them or update