Task Eventstreams in the scaffolder not being cleaned up properly

After upgrading our backstage repo to the most recent changes we have noticed that sometimes the eventstream crashes and then appears to stall the log output to the screen. From further investigation this appears to happen when there are stale eventstreams in the background from previous runs that are still ongoing.

From further investigation it also seems like it may occurs if the eventstream fails and it needs to reconnnect, the new event stream is opened and it gets the completed log but never updates the UI unless you refresh the page.

FWIW we are also still trying to figure out why start getting net::ERR_HTTP2_PROTOCOL_ERROR 200 errors.

Expected Behavior

The expected behavior is for there to ever only be one eventstream that we are listening to and for it to successfully reconnect to just that eventstream upon crashing, leading to the scaffolder to complete successfully even if ran multiple times in a row.

Also if an eventstream fails and reconnects it should accurately reflect the new eventstream’s logs in the UI.

Current Behavior

When you run the scaffolder multiple times in a row in one session, there are stale eventstreams that cause the scaffolder to look as if it hasn’t completed because it’s trying to read state from multiple streams. Once refreshing the page it works as intended.

If an eventstream fails and it creates a new one, it’s logs are not reflected in the UI

Possible Solution

Not sure but assuming it has something to do with react hooks/memo keeping stale information around

There may be both a FE and BE issue that need to be solved. Should we close the eventstreams https://github.com/backstage/backstage/blob/cbfa75bf69da90961daae517f28a106327ceb5d0/plugins/scaffolder/src/api.ts#L217 in this area of code by calling eventSource.close() on both error and completion events?

And then the FE issue seems to be that after reconnecting to an eventStream (because of a client side failure) it doesn’t update the log state in the UI.

Steps to Reproduce

1.Scaffold out a template in backstage 2. then without refreshing go back and scaffold out another one as well

For us this is pretty consistent in recreating the behaviour.

However after further debugging, we have also noticed that if the eventstream fails before completion and reconnects it does not update the new eventstream logs in the UI. Appearing stuck unless you refresh

Refreshing though seems to make things work as intended.

Context

We were in the process of updating our fork of backstage with upstream changes

Your Environment

• NodeJS Version (v12): 12.16.2
• Operating System and Version (e.g. Ubuntu 14.04): linux (this is happening in our dev environment but would happen locally as well I believe)
• Browser Information: chrome

[jrusso1020]

thanks @freben @benjdlambert let me give that a try and see if it helps!

[freben]

Yep, a keepalive comment message makes sense. It sounds good to be driven from the router as you initially tried, not the TaskWorker. However you probably need to put a res.flush(); after each write.