question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[welcome page][perf] Slow queries to generate welcome page

See original GitHub issue

How to reproduce the bug

  1. As user using Superset for years,
  2. Go to https://superset.d.musta.ch/superset/welcome/
  3. Welcome page show spinner very, very long time: kpFvNcc17A
  4. from MySQL Slow query log, I can see the query to get recent viewed items can take > 20 mins.

Environment

airbnb production

  • superset version: superset version: latest master

Checklist

Make sure to follow these steps before submitting your issue - thank you!

  • I have checked the superset logs for python stacktraces and included it here as text if there are any.
  • I have reproduced the issue with at least the latest released version of superset.
  • I have checked the issue tracker for the same issue and I haven’t found one similar.

Additional context

Add any other context about the problem here.

Issue Analytics

  • State:open
  • Created 2 years ago
  • Reactions:3
  • Comments:5 (1 by maintainers)

github_iconTop GitHub Comments

2reactions
cesar-loadsmartcommented, Sep 5, 2022

We’ve fixed our problem by creating a specific index for this query:

SELECT anon_1.dashboard_id AS anon_1_dashboard_id, anon_1.slice_id AS anon_1_slice_id, anon_1.action AS anon_1_action, anon_1.dttm AS anon_1_dttm, dashboards.slug AS dashboard_slug, dashboards.dashboard_title AS dashboards_dashboard_title, slices.slice_name AS slices_slice_name 
FROM (SELECT logs.dashboard_id AS dashboard_id, logs.slice_id AS slice_id, logs.action AS action, max(logs.dttm) AS dttm 
FROM logs 
WHERE logs.action IN ('explore', 'dashboard') AND logs.user_id = <USERID> AND logs.dttm > '2021-08-24T17:20:50.496891'::timestamp AND (logs.dashboard_id IS NOT NULL OR logs.slice_id IS NOT NULL) GROUP BY logs.dashboard_id, logs.slice_id, logs.action) AS anon_1 LEFT OUTER JOIN dashboards ON dashboards.id = anon_1.dashboard_id LEFT OUTER JOIN slices ON slices.id = anon_1.slice_id 
WHERE dashboards.dashboard_title != '' OR slices.slice_name != '' ORDER BY anon_1.dttm DESC 
 LIMIT 6

Our Superset became faster than it was released, today we have around 400 users and more than 100 published dashboards.

0reactions
jplanckeelcommented, Sep 15, 2022

Hello, we have created a purge for the log but it is slow because the logs table is constantly called. This has been fixed by creating an index on the dttm column of the logs table.

Can we add an index on next release ?

our PR for add purge on event logger 😉 https://github.com/apache/superset/pull/20636

CREATE INDEX "dttm_index"
ON "logs" ("dttm");
Read more comments on GitHub >

github_iconTop Results From Across the Web

Viewing slow queries | OpsCenter 6.7
Click the title bar of the Slow Queries panel. The Slow Queries performance details page appears. Sort the slow queries by any column...
Read more >
Use a slow query log - Product Documentation | ServiceNow
Use a slow query log · Navigate to All > System Diagnostics > Stats > Slow Queries · Open a query record for...
Read more >
How to Enable the Slow Query Log in MySQL® or MariaDB
In MySQL 5.6 and older, use the log-slow-queries variable instead of the slow-query_log_file variable. Create the /var/log/mysql-slow.log file ...
Read more >
Finding and Fixing Slow WordPress Database Queries
Slow SQL queries can crush your WordPress site's performance. Sometimes, slow queries are a result of poorly-formed SQL that never should ...
Read more >
New Slow Queries - pganalyze
DBAs and developers use pganalyze to identify the root cause of performance issues, optimize queries and to get alerts about critical issues.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found