[Proposal] Add allennlp integration

It would be nice to integrate with allennlp lib.

Motivation

Allennlp is a NLP library that is used in NLP researchers/practitioners including me. There is allentune, which is the library of parameter optimization for allennlp. By using allentune, users can search hyperparameters by random search. However, optuna provides some features which are not implemented in allentune (e.g. pruning). I think it would be useful that allennlp users can use optuna to search for hyperparameters.

Description

To support allennlp in optuna, I think we have to implement:

• Custom MetricTracker compatible with allennlp’s Trainer

  • http://docs.allennlp.org/master/api/training/metric_tracker/#metrictracker
  • MetricTracker performs early stopping

• Custom Trainer that inherits allennlp’s Trainer

  • latest allennlp doesn’t provide a way to use custom callback (https://github.com/allenai/allennlp/issues/3519)
    • (this topic is under discussion, so it could change an approach
  • So we may implement custom trainer inherits allennlp.training.Trainer

I want to discuss this in detail. Thanks,

[himkt]

I working on this in #1399.

[github-actions[bot]]

This issue has not seen any recent activity.