Automated learning algorithm overview

Contributors: @fkiraly, @mloning, @afzal442, @abdulelahsm, @sparkingdark

idea:

web data base for learning algorithms with fields for:

• class name,
• scitype (e.g. forecaster, classifier, etc.),
• original contributor,
• maintainer/current code owner,
• link to API reference,

optional additional fields:

• links to literature references,
• package dependency,
• code health/status (under development, mature, etc),
• link to example notebooks

use cases:

• search for suitable algorithms given a problem
• search for information about a given algorithm

reference example from mlr3 library:

• https://mlr3extralearners.mlr-org.com/articles/learners/learner_status.html
• https://mlr3extralearners.mlr-org.com/articles/learners/list_learners.html
• GitHub action to automatically generate table: https://github.com/mlr-org/mlr3extralearners/blob/main/.github/workflows/update_learner_table.yml

related code in sktime:

from sktime.utils import all_estimators
all_estimators(estimator_types="classifier")

https://github.com/alan-turing-institute/sktime/blob/ee7a06843a44f4aaec7582d847e36073a9ab0566/sktime/utils/__init__.py#L16

questions:

• integrate it with API reference: https://www.sktime.org/en/latest/api_reference.html (adding columns to that table)?

notes:

• make clear that not all algorithm are reference implementations, even though we link them with the original research paper, some may be re-implemented, adapted or interfaced from other packages
• encourage people to ping algorithm maintainers directly when they have questions about algorithm using the GitHub user name
• Adding col for maintainer using GH

[tobyhodges]

It looks like the format of the __author__ fields are not standardised across the different estimators in sktime: some are lists of multiple names (e.g. lines 1&2 above), some are lists with only one name (e.g. lines 3-5 above) and some are strings with a single name (e.g. line 6 above). Furthermore, we noticed that a few individual estimator functions have their own __author__ field, but that it is more common for the __author__ to be defined at the module level i.e. one __author__ for all the estimators in a given file. It might be a good idea for the sktime team to discuss this internally and agree on a strategy and format to use for author info across the entire library.

Anyway, for now I suggest you might use something like the below to handle the different cases and get some nicely written author info in your table:

from sktime.utils import all_estimators
from importlib import import_module
import csv

def process_author_info(author_info):
    '''turn a list of author names into a string. 
       Multiple author names will be separated by a comma, 
       with the final name always preceded by "&"'''
    if isinstance(author_info, list):
        if len(author_info) > 1:
            return ', '.join(author_info[:-1]) + ' & ' + author_info[-1]
        else:
            return author_info[0]
    else:
        return author_info

with open("author_list.csv", "w+", newline='') as csvfile:
     write = csv.writer(csvfile, delimiter=",")
     write.writerow(["Class_Name","Class_ID", "Authors"])
     for modname, modclass in all_estimators():
         try:
             write = csv.writer(csvfile, delimiter = ",")
             write.writerow([modname,modclass, 
                             process_author_info(modclass.__author__)]) # <--- apply the function here
         except AttributeError:
             try:
                 write = csv.writer(csvfile, delimiter = ",")
                 write.writerow([modname,modclass, 
                                 process_author_info(import_module(modclass.__module__).__author__)]) # <--- and here

             except AttributeError:
                 write = csv.writer(csvfile, delimiter = ",")
                 write.writerow([modname,modclass, "no author info"])

For the above example output, this gives:

| Class_Name                                | Class_ID                                                                                             | Authors                                                                               |
| ----------------------------------------- | ---------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------- |
| ARIMA                                     | <class 'sktime.forecasting.arima.ARIMA'>                                                             | Markus Löning & Hongyi Yang                                                           |
| AutoARIMA                                 | <class 'sktime.forecasting.arima.AutoARIMA'>                                                         | Markus Löning & Hongyi Yang                                                           |
| AutoCorrelationTransformer                | <class 'sktime.transformations.series.acf.AutoCorrelationTransformer'>                               | Afzal Ansari                                                                          |
| AutoETS                                   | <class 'sktime.forecasting.ets.AutoETS'>                                                             | Hongyi Yang                                                                           |
| BATS                                      | <class 'sktime.forecasting.bats.BATS'>                                                               | Martin Walter                                                                         |
| BOSSEnsemble                              | <class 'sktime.classification.dictionary_based._boss.BOSSEnsemble'>                                  | Matthew Middlehurst                                                                   |

[tobyhodges]

This looks so great! Excellent work folks 🚀 It was a pleasure to be (briefly) involved in this project 🙌