Update Overhaul: final QA and testing

After the last item of the second milestone is complete, we want to do a thorough testing phase before merging the update-overhaul branch into master.

Only after all testing is done, we merge the update-overhaul branch into master. We should probably also schedule this wisely, to make sure that we have sufficient dev and support coverage in the following days of the release.

Testing scenarios

For each of the install paths, follow the setup instructions and review the testing considerations below.

Install paths

Note: Before running a test, add your Github handle to the end of the task. After you complete it, check the box for the task.

• Test new installation on Voyager device (instructions) @jdeanwallace
• Test new installation on Hobbyist device (instructions) @jdeanwallace
• Test update path from an existing (legacy) TinyPilot Community installation on a Hobbyist device (instructions) @jdeanwallace
• Test update path from an existing TinyPilot Community installation (that’s already on the new update mechanism) using any device (instructions) @jdeanwallace
• Test downgrade path from TinyPilot Pro to Community using any device (instructions) @jotaen4tinypilot
• Test upgrade path to TinyPilot Pro from an existing TinyPilot Community installation (that’s already on the new update mechanism) using any device (instructions)

What to consider when testing?

Behaviour / integrity

• Make sure that we don’t break existing systems, e.g. by accidentally overriding existing app/system settings or configs @jotaen4tinypilot
• Double check that the new installation places all files and folders on the system, and that they have the correct permissions @jotaen4tinypilot
• Try out all functionality of TinyPilot to verify correct system setup @jotaen4tinypilot (testing all)
  • System > Update
  • System > Hostname
  • System > Video settings (testing one setting change is sufficient)
  • System > Logs
  • System > About
  • System > Power (testing restart is sufficient)
  • Actions > Paste
  • Actions > Screenshot
• Identify points of failure within the update flow and verify that the error behaviour is correct @jotaen4tinypilot

Final review

Code

• Do a final check that we have ported the entire behaviour from quick-install @jotaen4tinypilot
  • 👉 See https://github.com/tiny-pilot/tinypilot/issues/1078
• Final review of the mega PR @jotaen4tinypilot
• Final review of the new code, both the bundler one and in gatekeeper @jotaen4tinypilot
  • Are there open todos? Do we need to round up anything? Are we happy with the structure and naming?

[jotaen4tinypilot]

Pending https://github.com/tiny-pilot/tinypilot-pro/pull/541, the QA is all done.

As closing remark from my side, I’d feel confident enough to close this ticket, so I feel we have tested and considered everything quite thoroughly.

[jdeanwallace]

@jotaen4tinypilot

• I don’t have a (working) Hobbyist device at hand, because with mine the video capture chip is defunct. I’m not sure it makes sense for me to take over the two Hobbyist test items, because I wouldn’t be able to verify whether the video setup and configs are working.

Okay cool, I’ll take on these two Hobbyist test items.

• Regarding the “test upgrade path to Pro from Community”, I realized that I actually don’t know how to do this, because I usually just flash an SD card with a disk image for setting up a Pro device. But that’s obviously not an option in this case. Is there an official way to do this, or would I just download and run Pro’s quick-install script from the private repo?

I’ve sent you an email regarding the Pro installation steps.