Running androidTests in Travis

Summary:

I was thinking about how to write a load of espresso tests (see #586) for the app to try to prevent so many bugs cropping up.

To get good coverage, we’d need to be a logged in user. So the tests would be more standardised, I think it would be useful to have a test user the tests could login as.

Therefore I think it would be useful to have an account on beta commons (and maybe even on real/prod commons) which isn’t able to do anything - not even edit its own user talk page, or change its own password. Maybe:

• Create the account
• Upload a few test photographs (so the ‘home’ page is not blank)
• Get it ‘locked’ somehow

It also might be useful to have a similar test user which is a completely blank account (i.e. no uploads), as I could imagine that causing bugs in the app which we want to test for (#2154).

We could then safely commit its credentials to this repo so people can run the tests. There also might be a different way that doesn’t involve sharing credentials, if people have one please do suggest!

The alternate idea I had:

• Require developers to create their own test accounts according to some specific instructions and then
  • Log in to the app in the emulator and then run the test suite
  • Put their credentials in a file that is not commited to the repo (with some .gitignore rule)
• Somehow Travis has its own credentials (kept secret like Google Play keys) and does its testing like that?

The issue with this is I can imagine a lot of new contributors accidentally pushing their credentials to Github. While this would be bad for beta commons, this could be very bad on real/prod commons. The setup would add an extra barrier to entry for developers to start regularly using automated testing, which we want to encourage by making as easy as possible. I also have no idea how to do the bit about Travis.

Would you like to work on the issue?

I don’t really know how this would be done - hoping someone else here might. If not, I’m happy to post on the Commons helpdesk and see if we can get some answers.

Yes in terms of trying to add more android tests - although they’re so broad and don’t really conflict everyone is still free to (and should!) get writing tests. I’m not very good at it yet, but I’m trying to learn 😃

[maskaravivek]

need to be careful when running PRs that Travis shouldn’t reveal login details

Travis allows us to set encrypted environment variables which are not available to Pull requests. These values are available only to the original repo’s branches. The only nuance would be that the reviewer will have to run the tests locally to verify if the PR works.

@domdomegg As you pointed out in #2700, we should use mock values for our tests whenever possible. That way we can test the app is isolation. Anyways testing the backend APIs is not our intent. Just for some critical flows, we should use these credentials.

So basically, PRs won’t run the connectedCheck task and only master/other branch builds will run this task.

[madhurgupta10]

Fixed in https://github.com/commons-app/apps-android-commons/pull/4828