model is retrained everytime due to dynamic checkpoint names

Rasa version: Rasa 1.3.8 Commit 39300da9a5de7a3c7fd285519f9577466bfda8d5

Python version: 3.6.9

Operating system (windows, osx, …): osx

Issue: I trained my model, and running rasa shell everything works fine. But once I start rasa interactive, it starts training a model, even though I didn’t change anything. After Your Rasa model is trained and saved at ... and Type a message and press enter (press 'Ctr-c' to exit)., it continues with processing story blocks, which takes longer than training the model. (I aborted at this point.)

If I want do rasa train and rasa shell after that, the action server seems not accessible.

Error (including full traceback):

First, no error message, just unexpected behaviour. Here is a screenshot: Then, If I want do rasa train and rasa shell after that, I get this message:

rasa shell --debug                                                                                                                                       [16:59:55]
2019-10-15 17:00:53 DEBUG    rasa.model  - Extracted model to '/var/folders/yg/vcndsyk94tl492272h0x_wlm0000gp/T/tmpn8j0x0jm'.
2019-10-15 17:00:53 DEBUG    rasa.cli.utils  - Parameter 'endpoints' not set. Using default location 'endpoints.yml' instead.
2019-10-15 17:00:53 DEBUG    rasa.cli.utils  - Parameter 'credentials' not set. Using default location 'credentials.yml' instead.
2019-10-15 17:00:53 DEBUG    rasa.model  - Extracted model to '/var/folders/yg/vcndsyk94tl492272h0x_wlm0000gp/T/tmp2lsre1di'.
2019-10-15 17:00:54 INFO     root  - Connecting to channel 'cmdline' which was specified by the '--connector' argument. Any other channels will be ignored. To connect to all given channels, omit the '--connector' argument.
2019-10-15 17:00:54 DEBUG    sanic.root  - CORS: Configuring CORS with resources: {'/*': {'origins': [''], 'methods': 'DELETE, GET, HEAD, OPTIONS, PATCH, POST, PUT', 'allow_headers': ['.*'], 'expose_headers': None, 'supports_credentials': True, 'max_age': None, 'send_wildcard': False, 'automatic_options': True, 'vary_header': True, 'resources': {'/*': {'origins': ''}}, 'intercept_exceptions': True, 'always_send': True}}
2019-10-15 17:00:54 DEBUG    rasa.core.utils  - Available web server routes: 
/webhooks/rest                                     GET                            custom_webhook_CmdlineInput.health
/webhooks/rest/webhook                             POST                           custom_webhook_CmdlineInput.receive
/                                                  GET                            hello
2019-10-15 17:00:54 INFO     root  - Starting Rasa server on http://localhost:5005
2019-10-15 17:00:54 DEBUG    rasa.core.utils  - Using the default number of Sanic workers (1).
2019-10-15 17:00:54 INFO     root  - Enabling coroutine debugging. Loop id 5558218776.
2019-10-15 17:00:54 DEBUG    rasa.model  - Extracted model to '/var/folders/yg/vcndsyk94tl492272h0x_wlm0000gp/T/tmp6h47_0xh'.
2019-10-15 17:00:58 DEBUG    rasa.core.tracker_store  - Connected to InMemoryTrackerStore.
2019-10-15 17:00:58 DEBUG    rasa.core.lock_store  - Connected to lock store 'InMemoryLockStore'.
2019-10-15 17:00:58 DEBUG    rasa.model  - Extracted model to '/var/folders/yg/vcndsyk94tl492272h0x_wlm0000gp/T/tmp72_fr08t'.
2019-10-15 17:00:58 DEBUG    pykwalify.compat  - Using yaml library: /Users/johannes/miniconda3/envs/rasa/lib/python3.6/site-packages/ruamel/yaml/__init__.py
Bot loaded. Type a message and press enter (use '/stop' to exit): 
Your input ->  hi                                                                                                                                                                                                
Unable to start server
Traceback (most recent call last):
  File "/Users/johannes/miniconda3/envs/rasa/lib/python3.6/site-packages/sanic/server.py", line 778, in serve
    http_server = loop.run_until_complete(server_coroutine)
  File "uvloop/loop.pyx", line 1417, in uvloop.loop.Loop.run_until_complete
  File "uvloop/loop.pyx", line 1686, in create_server
OSError: [Errno 48] error while attempting to bind on address ('0.0.0.0', 5005): address already in use

Restarting my machine fixes this issue until I run rasa interactive again.

Command or request that led to error:

rasa train
rasa shell
rasa interactive  <-- problem here --/>

Content of configuration file (config.yml) (if relevant):

# Configuration for Rasa NLU.
# https://rasa.com/docs/rasa/nlu/components/
language: en
pipeline:
- name: "DucklingHTTPExtractor"
  # url of the running duckling server
  url: "http://localhost:8000"
  # dimensions to extract
  dimensions: ["duration", "number"]
  # allows you to configure the locale, by default the language is
  # used
  locale: "en_EN"
  # if not set the default timezone of Duckling is going to be used
  # needed to calculate dates from relative expressions like "tomorrow"
  timezone: "Europe/Berlin"
- name: "WhitespaceTokenizer"
- name: "RegexFeaturizer"
- name: "CRFEntityExtractor"
- name: "EntitySynonymMapper"
- name: "CountVectorsFeaturizer"
  analyzer: "char_wb"
  min_ngram: 1
  max_ngram: 4
- name: "EmbeddingIntentClassifier"

# Configuration for Rasa Core.
# https://rasa.com/docs/rasa/core/policies/
policies:
  - name: MemoizationPolicy
  - name: EmbeddingPolicy
    max_history: 5  # default: 5
    batch_strategy: "balanced" # alternative: "sequence"
    epochs: 30 # default: 1
    random_seed: 2019  # default: null
    evaluate_on_num_examples: 0 # default: 100
  - name: MappingPolicy
  - name: FormPolicy

Content of domain file (domain.yml) (if relevant):

%YAML 1.1
---
actions:
- rating_inform
- travel_inform
- utter_acknowledge_problem_badlink
- utter_ask_eval_binary_natural
- utter_ask_eval_binary_helpful
- utter_ask_eval_binary_recommend
- utter_ask_bot_eval
- utter_ask_bot_rating
- utter_ask_helpdevelopers
- utter_ask_passengercount
- utter_ask_rephrase
- utter_ask_reqmore
- utter_ask_addtravel
- utter_ask_should_have_done
- utter_ask_should_restart
- utter_ask_travel
- utter_ask_travelmode
- utter_ask_traveltime
- utter_ask_whattodo
- utter_ask_why_here
- utter_explain_why_offset_travel
- utter_general_about
- utter_general_affirm
- utter_general_deny
- utter_general_emotional_happy
- utter_general_emotional_resigned
- utter_general_emotional_surprised
- utter_general_emotional_wow
- utter_general_goodbye
- utter_general_greet
- utter_general_oos
- utter_general_oos_future
- utter_general_thankyou
- utter_general_yourewelcome
- utter_meta_acknowledge_privacy
- utter_meta_iamabot
- utter_meta_list_features
- utter_meta_privacy
- utter_sorry
entities:
- duration
- number
- travelmode
forms:
- rating_form
- travel_form
intents:
- general_convey_affirmation
- travel_convey_affirmation
- general_convey_denial
- general_convey_express_uncertainty
- general_convey_farewell
- general_convey_greeting
- general_convey_thankyou
- general_express_emotion_happy
- general_express_emotion_unhappy
- general_express_emotion_surprised
- general_inform_affirmation_all
- general_inform_alreadydoing
- general_inform_problem_badlink:
    triggers: utter_acknowledge_problem_badlink
- general_inform_problem_badreply
- general_inquire_clarification
- meta_inquire_capabilities:
    triggers: utter_meta_list_features
- meta_inquire_botchallenge:
    triggers: utter_meta_iamabot
- oos_inform:
    triggers: utter_general_oos
- oos_inquire:
    triggers: utter_general_oos
- oos-future_inform:
    triggers: utter_general_oos_future
- oos-future_inquire:
    triggers: utter_general_oos_future
- travel_inform
- travel_inform_distance
- travel_inform_duration
- travel_inquire_clarification
- travel_inquire_howto_offset:
    triggers: travel_form
- general_placeholder
slots:
  eval_binary_helpful:
    type: unfeaturized
  eval_binary_natural:
    type: unfeaturized
  eval_binary_recommend:
    type: unfeaturized
  passengercount:
    type: unfeaturized
  requested_slot:
    type: unfeaturized
  travelmode:
    type: unfeaturized
  traveltime:
    type: unfeaturized
  traveldistance:
    type: unfeaturized
templates:
  utter_acknowledge_problem_badlink:
  - text: Oh no! I am sorry the link doesn't work anymore. I'll let my developers
      know.
  utter_ask_bot_eval:
  - text: Anything you'd like my developers to know?
  utter_ask_bot_rating:
  - buttons:
    - payload: you are dreadful
      title: dreadful
    - payload: you are mediocre
      title: mediocre
    - payload: you are good
      title: good
    - payload: you must be a human!
      title: human-like
    text: How do you think my performance will be rated?
  utter_ask_eval_binary_helpful:
  - text: Was our conversation helpful to you / did you learn something by chatting
      with me?
  utter_ask_eval_binary_natural:
  - text: Would you say our conversation was natural / intuitive?
  utter_ask_eval_binary_recommend:
  - text: Would you ask others to chat with me?
  utter_ask_helpdevelopers:
  - text: Could you help me with something, too?
  utter_ask_passengercount:
  - text: How many people were in the car?
  utter_ask_rephrase:
  - text: Not sure if I got that right. Could you please rephrase this?
  utter_ask_reqmore:
  - text: What would you like to do now?
  - text: Can I help you with anything else?
  utter_ask_addtravel:
  - text: Would you like to add another travel mode?
  utter_ask_should_have_done:
  - text: Oh no. I am utterly confused now. What should I have said?
  utter_ask_should_restart:
  - text: Thanks. I will send this to my developers so they can improve me. For now,
      I suggest I wipe my memory and restart our conversation. I won't remember anything
      you've said. Is that ok?
  utter_ask_travel:
  - text: Did you travel recently, or are you planning to travel?
  utter_ask_travelmode:
  - buttons:
    - payload: By car
      title: car
    - payload: By plane
      title: airplane
    - payload: By train
      title: train
    text: By what means did you travel?
  utter_ask_traveltime:
  - text: And how long did you / will you travel with the {travelmode}?
  utter_ask_whattodo:
  - text: What would you like to do?
  utter_ask_why_here:
  - text: I see. Don't you want to reduce your greenhouse gas emissions?
  utter_explain_why_offset_travel:
  - text: Traveling, especially by airplane, usually emits greenhouse gases which
      are causing the ongoing climate change. If you cannot avoid these emissions,
      you can buy 'offsets', i.e. donations to projects that reduce greenhouse gases,
      e.g. by planting trees.
  utter_general_about:
  - text: I was built to help you reduce or offset your greenhouse gas emissions.
  utter_general_affirm:
  - text: 'Yes'
  utter_general_deny:
  - text: 'No'
  utter_general_emotional_happy:
  - text: :-)
  utter_general_emotional_resigned:
  - text: Too bad :-/
  utter_general_emotional_surprised:
  - text: Seriously?!
  utter_general_emotional_wow:
  - text: Wow!
  utter_general_goodbye:
  - text: Goodbye then!
  - text: Alright, bye!
  utter_general_greet:
  - text: Hello!
  - text: Hi
  utter_general_oos:
  - text: I am afraid I can't help you with that.
  - text: That's beyond the scope of my programming.
  utter_general_oos_future:
  - text: Good question! A future version of me might be able to answer this.
  utter_general_thankyou:
  - text: Thank you.
  - text: Thanks
  utter_general_yourewelcome:
  - text: You're welcome!
  - text: No problem.
  utter_meta_acknowledge_privacy:
  - text: Data collection is set to {allow_datacollection}.
  utter_meta_iamabot:
  - text: I am a bot, powered by Rasa.
  utter_meta_list_features:
  - text: At the moment, I can only help you offset CO2 emissions of your travels.
  utter_meta_privacy:
  - buttons:
    - payload: You can collect my data
      title: Yes, you can record and share my conversation.
    - payload: Please forget my data
      title: No, I'd like to keep this private.
    text: Can we record and publish the conversation you are about to have?
  utter_sorry:
  - text: Oh no, sorry!

Enable Squad Definition of Done:

rasa train will not retrain model if stories have not changed, even if they contain OR steps.

[akelad]

they still exist sadly, we don’t have a better solution yet

[wochinge]

@btotharye I unassigned it since you haven’t started yet. Feel free to pick it up again 😃