Testing: Add a `LiveParallelizable` that runs Live tests in parallel

We need to run tests sequentially when AZURE_TEST_MODE is Record or Playback. It would be nice to run tests in parallel when AZURE_TEST_MODE is Live but nUnit doesn’t allow that kind of flexibility out of the box. We should investigate adding a custom LiveParallelizable attribute at the assembly level.

[pakrym]

Very good point Kamil. I think the parallelization of test classes is technically achieveable.

[kasobol-msft]

@pakrym We won’t be able to go as far as parallelization per single test. That is probably true for live tests (at least for Storage) due to limitation you mentioned.

However, there’s a way to apply NUnit Parallelizable attribute in such a way that test classes are executed in parallel but within class methods go serially (there are few ways to achieve that). I would take ability to go parallel per class any day instead of current situation.

In my experiment I applied [Parallelizable(ParallelScope.Self)] on StorageTestBase. The issue I was facing is somewhere in this code https://github.com/Azure/azure-sdk-for-net/blob/52fb9b7628732e70a41dd057625619dcbaf31479/sdk/core/Azure.Core.TestFramework/src/ClientTestFixtureAttribute.cs#L58-L67 . Somehow it makes recording/recordingtransport or something in that vicinity being shared between sync/async mode which I couldn’t pin point. When I hacked this to generate only one permutation per tests I got pretty good results, i.e. whole assembly ran per-class-parallel and tests were green (in playback mode).