Improvements to FallbackClassifier

Description of Problem:

When moving the NLU fallback to the NLU side, we now overwrite the intent with nlu_fallback and assign that a confidence of 1.0 I’m not sure about the overwriting part and assigning it a confidence of 1.0 in the first place, but the main issue for me at the moment is that there’s no note of what the predicted intent was in the debug logs at the moment. We should be logging:

• that a fallback happened
• what intent was predicted originally and with what confidence (maybe even the full intent ranking? idk)

Bot loaded. Type a message and press enter (use '/stop' to exit): 
Your input ->  chitchat                                                                                                                                         
2020-07-28 15:29:34 DEBUG    rasa.core.tracker_store  - Creating a new tracker for id 'f0ffa4263bde49d5ba54eeb81d14640a'.
2020-07-28 15:29:34 DEBUG    rasa.core.processor  - Starting a new session for conversation ID 'f0ffa4263bde49d5ba54eeb81d14640a'.
2020-07-28 15:29:34 DEBUG    rasa.core.processor  - Action 'action_session_start' ended with events '[<rasa.core.events.SessionStarted object at 0x156491810>, <rasa.core.events.ActionExecuted object at 0x156700d10>]'.
2020-07-28 15:29:34 DEBUG    rasa.core.processor  - Current slot values: 
	concerts: None
	venues: None
2020-07-28 15:29:34 DEBUG    rasa.core.processor  - Received user message 'chitchat' with intent '{'name': 'nlu_fallback', 'confidence': 1.0}' and entities '[]'
2020-07-28 15:29:34 DEBUG    rasa.core.processor  - Logged UserUtterance - tracker now has 4 events.
2020-07-28 15:29:34 DEBUG    rasa.core.policies.memoization  - Current tracker state [None, None, None, {}, {'prev_action_listen': 1.0, 'intent_nlu_fallback': 1.0}]
2020-07-28 15:29:34 DEBUG    rasa.core.policies.memoization  - There is no memorised next action
2020-07-28 15:29:34 DEBUG    rasa.core.policies.rule_policy  - Current tracker state: [{}, {'prev_action_listen': 1.0, 'intent_nlu_fallback': 1.0}]
2020-07-28 15:29:34 DEBUG    rasa.core.policies.rule_policy  - There is a rule for next action 'utter_default'.
2020-07-28 15:29:34 DEBUG    rasa.core.policies.ensemble  - Predicted next action using policy_2_RulePolicy
2020-07-28 15:29:34 DEBUG    rasa.core.processor  - Predicted next action 'utter_default' with confidence 1.00.
2020-07-28 15:29:34 DEBUG    rasa.core.processor  - Action 'utter_default' ended with events '[BotUttered('default message', {"elements": null, "quick_replies": null, "buttons": null, "attachment": null, "image": null, "custom": null}, {"template_name": "utter_default"}, 1595942974.897126)]'.
2020-07-28 15:29:34 DEBUG    rasa.core.policies.memoization  - Current tracker state [None, None, {}, {'prev_action_listen': 1.0, 'intent_nlu_fallback': 1.0}, {'prev_utter_default': 1.0, 'intent_nlu_fallback': 1.0}]
2020-07-28 15:29:34 DEBUG    rasa.core.policies.memoization  - There is no memorised next action
2020-07-28 15:29:34 DEBUG    rasa.core.policies.rule_policy  - Current tracker state: [{}, {'prev_action_listen': 1.0, 'intent_nlu_fallback': 1.0}, {'prev_utter_default': 1.0, 'intent_nlu_fallback': 1.0}]
2020-07-28 15:29:34 DEBUG    rasa.core.policies.rule_policy  - There is a rule for next action 'action_listen'.
2020-07-28 15:29:34 DEBUG    rasa.core.policies.ensemble  - Predicted next action using policy_2_RulePolicy
2020-07-28 15:29:34 DEBUG    rasa.core.processor  - Predicted next action 'action_listen' with confidence 1.00.
2020-07-28 15:29:34 DEBUG    rasa.core.processor  - Action 'action_listen' ended with events '[]'.
2020-07-28 15:29:34 DEBUG    rasa.core.lock_store  - Deleted lock for conversation 'f0ffa4263bde49d5ba54eeb81d14640a'.
default message

[akelad]

i disagree vova, you want to be debugging your NLU model, and for that it’s important to see which intent it’s confused with so you can fix those issues in your training data

[wochinge]

I changed the priority to high as it’s related to testing / evaluation efforts which are most likely part of our OKRs for next cycle.