Implement metrics from doppyo

doppyo provides many metrics useful to climpred.

Metrics:

• contingency
• many scores derived from contingency (table)
• rank histogram
• reliability (diagram)
• check ROC
• check RPS

Quick preview of Reliability diagram for DPLE SST tutorial data:

Issues with doppyo metrics in climpred:

• some doppyo requirements: windspharm
• expects and returns xr.DataArray, climpred classes expects and return xr.Dataset
• some doppyo metrics return xr.Dataset with multiple data_vars (propose to return xr.Dataset or xr.DataArray with new dimension of previous data_vars)
• dim very similar to over_dims

Code base: https://github.com/csiro-dcfp/doppyo/blob/master/doppyo/skill.py

Trial notebook: https://gist.github.com/aaronspring/eef9496aad6516e6afcf3a89db35b0c9

Next steps:

• ask doppyo developers whether they would accept PRs for doppyo allowing xr.Datasets also and having extra dims if needed
• else implement in climpred
• how to implement into climpred: threshold metric similar to brier_score_threshold

I can imagine doppyo in a similar way as xskillscore providing metrics to climpred.

Thoughts on how to proceed?

[dougiesquire]

Thanks @aaronspring. I’ll try to start porting functions today and see how far I get.

[bradyrx]

Note also you can add to setup.py optional dependencies. So they could have windspharm as an optional dependency.

https://stackoverflow.com/questions/10572603/specifying-optional-dependencies-in-pypi-python-setup-py