Add documentation for how to test a Briefcase-packaged app

The following exchange occurred in our Gitter chat room. A formalized version of this should be incorporated into our documentation as a topic guide on testing:

@tmontes: Given that briefcase leans towards a src-based layout – which I like, for many reasons – how are people running tests against their application code? My previous experience, as is the case with briefcase 0.2, is mostly based on setuptools+setup.py where I’d install the application in editable mode (with pip install -e .) and from there, running tests would be “fine” because the application modules/packages would be importable.

@freakboy3742: There’s two options. The first is to change the sources definition. Although the default Briefcase project uses the src prefix for all it’s code, it doesn’t have to; you can specify any folder you want as a source. If you drop the src prefix from the default sources definition, you can use a “prefixless” source.

The other option is to use a test runner that supports different source prefixes. pytest, for example, has a pytest-pythonpath plugin that lets you add a python_paths option in the [tool:pytest] section of setup.cfg so you can add other prefixes to your source path.

---

We should include details for how to use unittest, pytest and as a stretch goal - trial.

[freakboy3742]

For those watching this ticket; pytest-briefcase has now been released to PyPI.

[freakboy3742]

@GenevieveBuckley pytest-briefcase is orthogonal to testpaths; testpaths defines where your tests are; pytest-briefcase makes sure that the code required by your app is as defined in your briefcase configuration. At present, it’s not much more than a tool for PYTHONPATH manipulation.

A briefcase project that isn’t using a src folder and contains a single app won’t really need pytest-briefcase (or, at least, won’t notice a significant change). However, if you’ve got an app that contains:

    myproject/
        src/
            myapp/
            otherapp/
        utils/
            otherstuff

and your briefcase configurations defines:

[tool.briefcase.app.myapp]
source = ['src/myapp', 'utils/otherstuff']

[tool.briefcase.app.secondapp]
source = ['src/secondapp', 'utils/otherstuff']

pytest-briefcase makes sure that the PYTHONPATH is correctly configured so that you can import myapp and import otherstuff when testing MyApp.

It currently runs entirely in the local development environment, assuming that your test environment contains all the relevant dependencies. I’d definitely like to extend it to making sure (or, at least, provide an ability to ensure) that the test environment has the dependencies specified by the Briefcase configuration. However, I’m not 100% sure if that might be better served as a feature of briefcase - e.g., a briefcase dev --no-run option, for example, that will enable you to ensure that dependencies are installed, but won’t actually run the app. That command could then be incorporated into your CI config in the same way that pip install -e . is often used for testing libraries.

Running the test suite with the actual packaged Python and environment would be an interesting long term goal, but there’s a bunch of complications. Firstly, the “deployed” environment won’t have pytest in it, so you’d need to find a way to inject it. Secondly, how do you run the test suite for iOS or Android apps? These are probably solvable problems, but might need a briefcase test option to allow for setting up the test environment.