Project Admin - dashboard load fails

Per chat w/ @xamanu @russdeffner - a few project admins are noticing a project loading error on the dashboard with 504 - Timeout error code.

Preliminary investigation shows this might be because of a nested loop situation.

This line here https://github.com/hotosm/tasking-manager/blob/develop/server/models/postgis/project.py#L348, which runs project summary for each project, is slowing down the process. Project summary has been updated in this recent release to do average time computations:

....

 "totalValidationTime": "22:47:45.990745",
  "totalTimeSpent": "11:33:46.122691",
  "averageMappingTime": "0:00:33.326294",
  "averageValidationTime": "0:08:17.363636",
  "status": "PUBLISHED"

Internally this fetches related tasks from the project table, iterates over the results and adds up time spent for each of the task. Now imagine this happening for 466 projects from @russdeffner - it is a loop within a loop

all projects for a user -> for each project:
    fetch tasks: ->  iterate over task list and add up time:

This is a classic case of a nested loop that increases the time complexity of a program (O(n²)). When an admin has more projects, this complexity is enhanced resulting in a timeout error.

How to solve?

1. Remove time calculations from the project summary. Include time estimates in a different endpoint
2. Retain time calculations within the project summary, but sum up time values from the column for validation & mapping. A limitation is the field type (varchar).
3. Retain time calculation with project summary, add a centroid field to the project dto. Now for project fetch within the admin dashboard, invoke project dto function instead of the project summary function. Not able to tell immediately if this will impact other endpoints. But @russdeffner & I tried it briefly this evening and the frontend map rendering fails.

Next actions:

• Try solution 2
• If solution 2 fails, look at solution 1 (separate out time calculations)

[Vaomatua]

@ramyaragupathy Yes, the dashboard now loads in about 5 seconds for me. Thank you for fixing this.

[russdeffner]

Yep, works for me as well (on mobile).

On Jun 5, 2019, at 10:16 AM, Emmor [e-mail] wrote:

@ramyaragupathy Yes, the dashboard now loads in about 5 seconds for me. Thank you for fixing this.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.