Global Async Queries doesn't work

Hi. I tried to enable GAQ feature with apache/superset:latest docker image. After I enabled it I experienced some dashboard stopped working and all I see is white screen. When I checked network activity I saw too much async request and they weren’t stoping and returing emtpy results.

Expected results

Dashboard should load faster and async

Actual results

Charts and dashboards aren’t loading. Also see this exception in worker

worker_1    | [2021-01-26 10:16:36,308: ERROR/ForkPoolWorker-11] Task load_explore_json_into_cache[069996ac-d5d9-4c29-a6e4-c99e3faf6899] raised unexpected: NoSuchColumnError("Could not locate column in row for column 'table_columns.id'")
worker_1    | Traceback (most recent call last):
worker_1    |   File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 412, in trace_task
worker_1    |     R = retval = fun(*args, **kwargs)
worker_1    |   File "/app/superset/app.py", line 116, in __call__
worker_1    |     return task_base.__call__(self, *args, **kwargs)
worker_1    |   File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 704, in __protected_call__
worker_1    |     return self.run(*args, **kwargs)
worker_1    |   File "/app/superset/tasks/async_queries.py", line 108, in load_explore_json_into_cache
worker_1    |     raise exc
worker_1    |   File "/app/superset/tasks/async_queries.py", line 84, in load_explore_json_into_cache
worker_1    |     payload = viz_obj.get_payload()
worker_1    |   File "/app/superset/viz.py", line 476, in get_payload
worker_1    |     columns = set(self.datasource.column_names)
worker_1    |   File "/app/superset/connectors/base/models.py", line 145, in column_names
worker_1    |     return sorted([c.column_name for c in self.columns], key=lambda x: x or "")
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/attributes.py", line 287, in __get__
worker_1    |     return self.impl.get(instance_state(instance), dict_)
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/attributes.py", line 723, in get
worker_1    |     value = self.callable_(state, passive)
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/strategies.py", line 760, in _load_for_state
worker_1    |     session, state, primary_key_identity, passive
worker_1    |   File "<string>", line 1, in <lambda>
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/strategies.py", line 902, in _emit_lazyload
worker_1    |     .with_post_criteria(set_default_params)
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/ext/baked.py", line 544, in all
worker_1    |     return list(self)
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 100, in instances
worker_1    |     cursor.close()
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
worker_1    |     with_traceback=exc_tb,
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
worker_1    |     raise exception
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 80, in instances
worker_1    |     rows = [proc(row) for row in fetch]
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 80, in <listcomp>
worker_1    |     rows = [proc(row) for row in fetch]
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 524, in _instance
worker_1    |     tuple([row[column] for column in pk_cols]),
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 524, in <listcomp>
worker_1    |     tuple([row[column] for column in pk_cols]),
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/result.py", line 686, in _key_fallback
worker_1    |     replace_context=err,
worker_1    |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
worker_1    |     raise exception
worker_1    | sqlalchemy.exc.NoSuchColumnError: "Could not locate column in row for column 'table_columns.id'"

Screenshots

How to reproduce the bug

1. Create superset and celery environment with apache/superset:latest image
2. Enable GAQ feature and configure secret.

Environment

• superset version: latest
• python version: 3.7
• mysql 5.7.32
• redis 6.0.9

Checklist

Make sure to follow these steps before submitting your issue - thank you!

• I have checked the superset logs for python stacktraces and included it here as text if there are any.
• I have reproduced the issue with at least the latest released version of superset.
• I have checked the issue tracker for the same issue and I haven’t found one similar.

[robdiciuccio]

I’ve been able to reproduce the issue using Celery prefork concurrency. It seems to be the same issue documented here: https://github.com/apache/superset/issues/10530

Continuing to investigate.

[iercan]

@robdiciuccio Here is my final report. Chart load problem gone after I set GLOBAL_ASYNC_QUERIES_POLLING_DELAY from 500 to 1500. I observed dashboard load times with GAQ enabled and disabled. It seems while GAQ disabled dashboards loads faster. I believe it is because I had to increase pooling delay and our dashboards usually have 5-10 charts. As a result we decided to go on with GAQ disabled for now. I’d glad test it again after some improvements. Thanks