Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

How to execute tests against only 1 shard of X shards?

See original GitHub issue

How can I have marathon split up tests in a deterministic way? For example, let’s say a module has 100 tests, I want to split them into 4 different shards, and run tests against one of those shards (so only 25 of the tests)?

previously I did this with Spoon using --numShards 4 --shard-index 0

Issue Analytics

State:
Created 3 years ago
Comments:14 (6 by maintainers)

Top GitHub Comments

1reaction

Malinskiycommented, Apr 25, 2020

You can extract the list of test same way we’re doing in marathon: https://github.com/linkedin/dex-test-parser. If you don’t want to develop a separate tool for that, it’s possible to reuse your gradle build code and create a task that outputs the test methods

1reaction

Malinskiycommented, Feb 3, 2021

@ZakTaccardi marathon is not a platform, it’s just a test runner. vendor-android assumes that devices are available via adb server. If you have the resources to spin up emulators I suggest creating a basic version of mobile farm and connect the devices using adb connect TCP/IP.

Spinning up emulators locally on the agent doesn’t scale since you can’t have a huge agent to spawn 50+ emulators. And this is exactly the scenario that marathon is targeted: having 50 or even hundreds of devices that in parallel execute your tests.

I’ve built android-farm in my previous company for the purpose of infinite horizontal scaling. Take a look if you’re interested in this topic. https://github.com/agoda-com/android-farm

The approach with shards might work for small/medium setups but cannot be scaled properly. The reason behind is you’d need to setup 50+ jobs in CI. Each of them might fail and these runs are not connected and do not share devices between each other. Compare that with a pool of devices for a single run in marathon: even if a device will fail, the test run still continues since we have devices to spare. Reporting is also significantly simplified: you get 1 report rather than 50. Also, there is a problem with test runs being unbalanced: it is impossible to separate the shard in such a way that all the test runs finish at the same point of time. If they don’t finish at the same time then you’d be wasting resources of the finished test runs.