Consolidate smoke tests and best practices with DPG partner tools

Summary

As part of the DPG effort, we are providing tools and guidance to our service team partners:

• To enable them to create a new SDK in the repo for their service, that builds and passes CIs in 10 minutes
• That has all the information they need to follow Azure SDK best practices to create tests, samples, and other customer-facing documents in a matter of days

In order for our partners to be successful, we need the following:

• Client code our partners generate should be generated using the latest version of autorest in the repo, otherwise partners could ask to review a library that doesn’t follow the latest guidelines.
• The new SDK projects our partners create with our tools should build and pass CI at all times, otherwise our partners will need additional support to use our tools, when the tools should be self-serve.
• There should be one source of guidance for creating tests, samples and other docs exists in one place, since providing the same guidance in multiple places can cause the instructions to differ and this can create confusion for partners about which guidance they should follow.

Current problems

• The template project is not being updated automatically when a new generator version is applied to the repo

  This is because only the files under /sdk are regenerated when the generator version is updated in the repo

• The output of the generated project is not currently being smoke tested, so we don’t know whether it will build and pass CIs

  This is because smoke tests are running on an HLC client

• Guidance for creating tests, samples, and other docs exists under both /eng/templates and /sdk/template

  For example:

  • Tests: /eng/templates | /sdk/template
  • Samples: /eng/templates | /sdk/template

Success Criteria

• The client code generated by the quickstart tool is generated using the latest version of autorest in the repo
• The project generated by the quickstart tool builds and produces a library that passes CI
• The guidance for creating tests, samples and other docs exists in one place, and isn’t duplicated in multiple areas of the repo. This is related to https://github.com/azure/azure-sdk-for-net/issues/27770, in that all the guidance under /doc/Data Plane Code Generation should refer to whatever is being created and smoke tested.

Notes

1. This issue was previously titled: Figure out the story around how sdk/template and eng/templates/Azure.ServiceTemplate.Template interact, with the description

We are adding template files as part of https://github.com/azure/azure-sdk-for-net/issues/24495, but having two templates is going to cause a lot of confusion about which is the source of truth, as well as maintenance issues, etc.

We need a story to merge these two things into one.

1. This issue replaces:

• https://github.com/Azure/azure-sdk-for-net/issues/27397
• https://github.com/Azure/azure-sdk-for-net/issues/26590

[weshaggard]

I would like to combine these that is why @annelo-msft opened this issue but when we do combine, we should ensure we can continue to use it for testing out engsys stuff.

[weshaggard]

That seems like a good direction to me. As for whether or not it is possible I’m not sure about those particular details. I mostly would like for us to only have one template instead of multiple.