Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

retraining DIET does not imply retraining TED?

See original GitHub issue

I used the main branch, created a new project via rasa init and ran rasa train. After that I added some lookup data to my nlu.yml file. I then asked Rasa to train again.

> rasa train
2021-10-19 11:22:35 INFO     rasa.engine.training.hooks  - Starting to train component 'RegexFeaturizerGraphComponent'.
2021-10-19 11:22:35 INFO     rasa.engine.training.hooks  - Finished training component 'RegexFeaturizerGraphComponent'.
2021-10-19 11:22:35 INFO     rasa.engine.training.hooks  - Starting to train component 'LexicalSyntacticFeaturizerGraphComponent'.
2021-10-19 11:22:35 INFO     rasa.engine.training.hooks  - Finished training component 'LexicalSyntacticFeaturizerGraphComponent'.
2021-10-19 11:22:35 INFO     rasa.engine.training.hooks  - Starting to train component 'CountVectorsFeaturizerGraphComponent'.
2021-10-19 11:22:35 INFO     rasa.engine.training.hooks  - Finished training component 'CountVectorsFeaturizerGraphComponent'.
2021-10-19 11:22:35 INFO     rasa.engine.training.hooks  - Starting to train component 'CountVectorsFeaturizerGraphComponent'.
2021-10-19 11:22:35 INFO     rasa.engine.training.hooks  - Finished training component 'CountVectorsFeaturizerGraphComponent'.
2021-10-19 11:22:35 INFO     rasa.engine.training.hooks  - Starting to train component 'MemoizationPolicyGraphComponent'.
2021-10-19 11:22:35 INFO     rasa.engine.training.hooks  - Finished training component 'MemoizationPolicyGraphComponent'.
2021-10-19 11:22:35 INFO     rasa.engine.training.hooks  - Starting to train component 'RulePolicyGraphComponent'.
2021-10-19 11:22:35 INFO     rasa.engine.training.hooks  - Finished training component 'RulePolicyGraphComponent'.
2021-10-19 11:22:35 INFO     rasa.engine.training.hooks  - Starting to train component 'TEDPolicyGraphComponent'.
2021-10-19 11:22:35 INFO     rasa.engine.training.hooks  - Finished training component 'TEDPolicyGraphComponent'.
2021-10-19 11:22:35 INFO     rasa.engine.training.hooks  - Starting to train component 'UnexpecTEDIntentPolicyGraphComponent'.
2021-10-19 11:22:35 INFO     rasa.engine.training.hooks  - Finished training component 'UnexpecTEDIntentPolicyGraphComponent'.
2021-10-19 11:22:36 INFO     rasa.engine.training.hooks  - Starting to train component 'DIETClassifierGraphComponent'.
Epochs: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [00:19<00:00,  5.03it/s, t_loss=1.17, i_acc=1]
2021-10-19 11:22:56 INFO     rasa.engine.training.hooks  - Finished training component 'DIETClassifierGraphComponent'.
2021-10-19 11:22:56 INFO     rasa.engine.training.hooks  - Starting to train component 'EntitySynonymMapperGraphComponent'.
2021-10-19 11:22:56 INFO     rasa.engine.training.hooks  - Finished training component 'EntitySynonymMapperGraphComponent'.
2021-10-19 11:22:56 INFO     rasa.engine.training.hooks  - Starting to train component 'ResponseSelectorGraphComponent'.
2021-10-19 11:22:56 INFO     rasa.nlu.selectors.response_selector  - Retrieval intent parameter was left to its default value. This response selector will be trained on training examples combining all retrieval intents.
2021-10-19 11:22:56 INFO     rasa.engine.training.hooks  - Finished training component 'ResponseSelectorGraphComponent'.
Your Rasa model is trained and saved at 'models/20211019-112234.tar.gz'.`

A few things are strange about this.

Technically, the DIETClassifier doesn’t need to retrain when there’s new lookup values, right? The training data technically did not change.
More strangely, it seems like the DIETClassifier updated. So why didn’t the Policy models retrain? With a new NLU model surely we need a new TEDPolicy?

I’m assuming TED isn’t retraining because there’s no progress bar. Is this accurate in 3.x?

Issue Analytics

State:
Created 2 years ago
Comments:9 (9 by maintainers)

Top GitHub Comments

1reaction

wochingecommented, Oct 19, 2021

Thanks for testing it at this early stage @koaning 🙌🏻

0reactions

wochingecommented, Oct 19, 2021

Technically, the DIETClassifier doesn’t need to retrain when there’s new lookup values, right? The training data technically did not change.

This is true. To fix this we’d need to separate the lookup tables from the TrainingData object which we considered out of scope of the revamp as this has larger implications 😬