Model ID changes after retrain for 3.0 RC1

Rasa Open Source version

3.0.0rc1

Rasa SDK version

No response

Rasa X version

No response

Python version

3.8

What operating system are you using?

OSX

What happened?

• Train model with rasa train
• Run server with rasa run
• POST events to /webhooks/rest/webhook to process them
• GET tracker from /conversations/{{conversation_id}}/tracker to inspect stored events
• Stop server
• Retrain model (using rasa train)
• Run server (rasa run)
• POST events to /webhooks/rest/webhook to process them
• GET tracker from /conversations/{{conversation_id}}/tracker to inspect stored events

Model IDs in the tracker has changed from 55b155688cba4756ba05b0cd5b753034 to 45320b316d46487baf4b9795b073f3f4, also previous messages are not there anymore. If no retraining is done, it works fine and model ID stays the same and previous messages are there.

Model IDs should stay the same after retrains.

This test was under the QA plan for RC1.

Command / Request

No response

Relevant log output

No response

[usc-m]

What do you mean by this exactly?

Events from before the retrain are persisted and have the original model_id, and then events processed after the retrain have the model_id of the new model (and are also persisted). So model_ids are never changed once set on event objects but can change for the model themselves (my original understanding was that model IDs didn’t change when you retrained a model - like it was associated to the config in some way - which is how I wrote the test case)

[usc-m]

Yep, looks like it does change after retrain - but I can’t replicate the loss of events. If I use a persistent tracker store (i.e. SQLite) the events from the model before are persisted, as are the ones after the retrain. @joejuzl is this correct behaviour? If so I’ll change the test case on the QA pages to make it more clear