question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

ngclient testing: client workflow sequences

See original GitHub issue

Description of issue or feature request:

If needed, narrow down the scope of this issue and spawn separate ones for top-level metadata update, delegations metadata update, Fetching targets.

edit: This issue deals with top-level-metadata updates. Delegated roles and targets will be covered #1641 #1642 #1643 .

Following the high-level steps described in #1579 implement tests for the various client workflow sequences:

  • Tests scenarios from specification (Can we mark these to be our conformance tests?)
  • Specific sequences the client should be able to handle go here
  • Extracted useful scenarios from legacy tests
  • Specific python-tuf sequences (for ex. loading and validating local metadata)

Decide what we want to test sometimes and what should be always included:

  • errors / no errors
  • what metadata is downloaded?
  • what metadata is read from disk?
  • what metadata is on disk afterwards?
  • are the currently loaded metadata versions correct?
  • (some of the same bullet points repeated for targets)
  • ? more ?

The tests should be RepositorySimulator based.

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:6 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
sechkovacommented, Nov 10, 2021

I opened two follow up issues for testing the rest of the steps following the client workflow from the specification: #1641 #1642. We can close this one.

0reactions
jkucommented, Nov 10, 2021

The new set of tests (test_updater_top_level_update.py) are in my opinion the state of the art now:

  • tests are reasonably short and readable
  • it’s easy to see what individual tests actually check

If we can document the decision making process that lead to these choices (situation X -> should test for Y) maybe in the test module docstring or somewhere that would be good… but if there are no generic rules and we just need to look at individual tests, then I guess the current examples are good examples to look at.

Can we close this or are there more sub issues that need to be filed first?

Read more comments on GitHub >

github_iconTop Results From Across the Web

Testing Multiple Services using a Workflow - wcfstorm
In this scenario we will invoke 2 methods in sequence using workflow. (Note: workflows are supported on version 3.0 or higher). Steps: Click...
Read more >
Main Sequence and Client Sequence File - NI
The Main sequence initiates the tests on a UUT. The process model defines what is constant about the testing process, and Main sequences...
Read more >
Automate Client Onboarding with ConvertKit Forms and ...
By using upcoach automation workflows, add your clients on specific actions to your ConvertKit forms and sequences.
Read more >
Create and edit sequences - HubSpot Knowledge Base
Workflows are another automation tool that you can use to automatically enroll records in a series of actions. When creating your sequence, ...
Read more >
Modifying a compound test - IBM
A compound test is a testing workflow comprising smaller tests and other test elements in a certain sequence. You might want to order...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found