Tests should run without network access and be consistent in PyPi

Is your feature request related to a problem? Please describe. I package various Azure SDK components in Fedora Linux for use with azure-cli, and one of my frustrations is around running tests. During RPM builds, no network access is allowed. However, many of the SDK tests require network access or they have a VCR cassette that is outdated. This prevents me from running tests.

I’ve reviewed how Arch and Debian packagers are handling this challenge and many of them are disabling many tests or simply not testing at all. To make matters worse, some of the PyPi packages for certain SDK components contain tests, and some do not. Some of them contain the VCR recordings for tests, and others do not. This requires me to get the source code from the git repository so I can be sure I have all of the tests and recordings. However, I must download the entire repository (> 100MB) just to get the 1-2MB of code that I really care about.

Some tests, such as the ones for azure-mgmt-network, require me to run docker during tests, which includes downloading a remote container.

Describe the solution you’d like I’d like to see:

• SDK tests run without network dependencies
• Tests and VCR recordings either included or excluded consistently in PyPi packages

Describe alternatives you’ve considered Sometimes I have to disable some or all of the tests for a particular part of the SDK. For some, such as azure-mgmt-network, I must disable all of the tests to build the package successfully. This creates issues later when I get bugs from users or when I package azure-cli and its tests break when importing command modules.

[msyyc]

We do include tests in sdist but recording files are not included since current recording files are .json files after migration to testproxy. We will update the template to ensure the recording files are included in sdist.

[l0lawrence]

Hi @major, thanks for the feedback. We will investigate asap.