Consistently getting MySQLdb._exceptions.OperationalError: (1050, "Table 'xxxx' already exists") with different tables

I’m using parallel Optuna with a mysqldb in the rl-baselines3-zoo project. Normally this works fine, but with one environment I keep getting messages like “MySQLdb._exceptions.OperationalError: (1050, “Table ‘trial_intermediate_values’ already exists”)” or “MySQLdb._exceptions.OperationalError: (1050, “Table ‘trial_heartbeats’ already exists”)”. Per the downstream issue (https://github.com/DLR-RM/rl-baselines3-zoo/issues/177), this looks to be an issue with Optuna that I’ve been unable to debug after extensive effort. I’d be willing to contribute a PR for this is the precise nature of the issue can be determined. Curiously, all trials go to tuning after this error occurs.

Running like normal, or at least producing an error message that allows for debugging.

Environment

• Optuna version: tried 3.9.0 and 3.10.0
• Python version: 3.7.9
• OS: Ubuntu 20.04
• (Optional) Other libraries and their versions:

Error messages, stack traces, or logs

One for log is in https://github.com/DLR-RM/rl-baselines3-zoo/issues/177

The other is:

========== LunarLanderContinuous-v2 ==========
Seed: 3071150484
Default hyperparameters for environment (ones being tuned will be overridden):
OrderedDict([('batch_size', 64),
             ('ent_coef', 0.01),
             ('gae_lambda', 0.98),
             ('gamma', 0.999),
             ('n_envs', 16),
             ('n_epochs', 4),
             ('n_steps', 1024),
             ('n_timesteps', 1000000.0),
             ('policy', 'MlpPolicy')])
Using 4 environments
Overwriting n_timesteps with n=540000000
Optimizing hyperparameters
Sampler: tpe - Pruner: median
Traceback (most recent call last):
  File "/home/j_k_terry/rl-baselines3-zoo/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1800, in _execute_context
    cursor, statement, parameters, context
  File "/home/j_k_terry/rl-baselines3-zoo/venv/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 717, in do_execute
    cursor.execute(statement, parameters)
  File "/home/j_k_terry/rl-baselines3-zoo/venv/lib/python3.7/site-packages/MySQLdb/cursors.py", line 206, in execute
    res = self._query(query)
  File "/home/j_k_terry/rl-baselines3-zoo/venv/lib/python3.7/site-packages/MySQLdb/cursors.py", line 319, in _query
    db.query(q)
  File "/home/j_k_terry/rl-baselines3-zoo/venv/lib/python3.7/site-packages/MySQLdb/connections.py", line 259, in query
    _mysql.connection.query(self, query)
MySQLdb._exceptions.OperationalError: (1050, "Table 'trial_intermediate_values' already exists")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "train.py", line 192, in <module>
    exp_manager.hyperparameters_optimization()
  File "/home/j_k_terry/rl-baselines3-zoo/utils/exp_manager.py", line 678, in hyperparameters_optimization
    direction="maximize",
  File "/home/j_k_terry/rl-baselines3-zoo/venv/lib/python3.7/site-packages/optuna/study/study.py", line 1136, in create_study
    storage = storages.get_storage(storage)
  File "/home/j_k_terry/rl-baselines3-zoo/venv/lib/python3.7/site-packages/optuna/storages/__init__.py", line 31, in get_storage
    return _CachedStorage(RDBStorage(storage))
  File "/home/j_k_terry/rl-baselines3-zoo/venv/lib/python3.7/site-packages/optuna/storages/_rdb/storage.py", line 183, in __init__
    models.BaseModel.metadata.create_all(self.engine)
  File "/home/j_k_terry/rl-baselines3-zoo/venv/lib/python3.7/site-packages/sqlalchemy/sql/schema.py", line 4747, in create_all
    ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
  File "/home/j_k_terry/rl-baselines3-zoo/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 3107, in _run_ddl_visitor
    conn._run_ddl_visitor(visitorcallable, element, **kwargs)
  File "/home/j_k_terry/rl-baselines3-zoo/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 2110, in _run_ddl_visitor
    visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
  File "/home/j_k_terry/rl-baselines3-zoo/venv/lib/python3.7/site-packages/sqlalchemy/sql/visitors.py", line 524, in traverse_single
    return meth(obj, **kw)
  File "/home/j_k_terry/rl-baselines3-zoo/venv/lib/python3.7/site-packages/sqlalchemy/sql/ddl.py", line 850, in visit_metadata
    _is_metadata_operation=True,
  File "/home/j_k_terry/rl-baselines3-zoo/venv/lib/python3.7/site-packages/sqlalchemy/sql/visitors.py", line 524, in traverse_single
    return meth(obj, **kw)
  File "/home/j_k_terry/rl-baselines3-zoo/venv/lib/python3.7/site-packages/sqlalchemy/sql/ddl.py", line 895, in visit_table
    include_foreign_key_constraints,  # noqa
  File "/home/j_k_terry/rl-baselines3-zoo/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1286, in execute
    return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
  File "/home/j_k_terry/rl-baselines3-zoo/venv/lib/python3.7/site-packages/sqlalchemy/sql/ddl.py", line 78, in _execute_on_connection
    self, multiparams, params, execution_options
  File "/home/j_k_terry/rl-baselines3-zoo/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1384, in _execute_ddl
    compiled,
  File "/home/j_k_terry/rl-baselines3-zoo/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1843, in _execute_context
    e, statement, parameters, cursor, context
  File "/home/j_k_terry/rl-baselines3-zoo/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 2024, in _handle_dbapi_exception
    sqlalchemy_exception, with_traceback=exc_info[2], from_=e
  File "/home/j_k_terry/rl-baselines3-zoo/venv/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 207, in raise_
    raise exception
  File "/home/j_k_terry/rl-baselines3-zoo/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1800, in _execute_context
    cursor, statement, parameters, context
  File "/home/j_k_terry/rl-baselines3-zoo/venv/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 717, in do_execute
    cursor.execute(statement, parameters)
  File "/home/j_k_terry/rl-baselines3-zoo/venv/lib/python3.7/site-packages/MySQLdb/cursors.py", line 206, in execute
    res = self._query(query)
  File "/home/j_k_terry/rl-baselines3-zoo/venv/lib/python3.7/site-packages/MySQLdb/cursors.py", line 319, in _query
    db.query(q)
  File "/home/j_k_terry/rl-baselines3-zoo/venv/lib/python3.7/site-packages/MySQLdb/connections.py", line 259, in query
    _mysql.connection.query(self, query)
sqlalchemy.exc.OperationalError: (MySQLdb._exceptions.OperationalError) (1050, "Table 'trial_intermediate_values' already exists")
[SQL: 
CREATE TABLE trial_intermediate_values (
	trial_intermediate_value_id INTEGER NOT NULL AUTO_INCREMENT, 
	trial_id INTEGER NOT NULL, 
	step INTEGER NOT NULL, 
	intermediate_value FLOAT NOT NULL, 
	PRIMARY KEY (trial_intermediate_value_id), 
	UNIQUE (trial_id, step), 
	FOREIGN KEY(trial_id) REFERENCES trials (trial_id)
)
]
(Background on this error at: https://sqlalche.me/e/14/e3q8)

[jkterry1]

@nzw0301 could you make this no-stale? I’m still working on this

[github-actions[bot]]

This issue was closed automatically because it had not seen any recent activity. If you want to discuss it, you can reopen it freely.