question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Stories with only a start checkpoint and an end checkpoint are ignored during training

See original GitHub issue

Rasa version: 1.1.8

Python version: 3.6

Operating system (windows, osx, …): Ubuntu (using WSL)

Issue: Stories with only a start checkpoint and an end checkpoint are ignored during training. The story looks like this:

## within form interest blank
> within_form_apartment_interest
> back_from_within_form_apartment_interest

Context: To give you some context. In order to deal with different questions that could happen during a form without having to create the full story for each of them, I have created different stories connected by two checkpoints within_form_apartment_interest and back_from_within_form_apartment_interest:

  • A story to start the form (in practice, many stories)
  • A story after the form (in pracice, many stories as well)
  • Several stories within the form. For example, a user might ask why we need this field, or ask another question.

Questions are handled with the “within form stories” as:

## explain - explain why we need this field
> within_form_apartment_interest
* explain
  - action_explain
  - form_apartment_interest
> back_from_within_form_apartment_interest

## ask_available - data is set
> within_form_apartment_interest
* ask_available
  - action_answer_available
  - form_apartment_interest
> back_from_within_form_apartment_interest

## within form interest blank (happy path)
> within_form_apartment_interest
> back_from_within_form_apartment_interest

The problem is that I also need one story where nothing wrong happens for the “happy path” to work. Unfortunately, this story only contains checkpoints and is not taken into account.

These checkpoints are connected to the following stories (form beginning and after form):

## beginning of form
  ...
* some_intent
      - form_apartment_interest
      - form{"name": "form_apartment_interest"}
  > within_form_apartment_interest

## after form (there are multiple of these)
  > back_from_within_form_apartment_interest
      - form{"name": null}
      - utter_thanks

Error (including full traceback): N/A

Command or request that led to error: N/A

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:12 (11 by maintainers)

github_iconTop GitHub Comments

3reactions
tmbocommented, Aug 23, 2019

ok, I guess the question is more: do we actually need these checkpoint only stories. I mean you could very well, instead of having this “glue story” just use the checkpoint directly.

so instead of

## ask_available - data is set
> within_form_apartment_interest
* ask_available
  - action_answer_available
  - form_apartment_interest
> back_from_within_form_apartment_interest

## within form interest blank (happy path)
> within_form_apartment_interest
> back_from_within_form_apartment_interest

## after form (there are multiple of these)
  > back_from_within_form_apartment_interest
      - form{"name": null}
      - utter_thanks

it would just be

## ask_available - data is set
> within_form_apartment_interest
* ask_available
  - action_answer_available
  - form_apartment_interest
> back_from_within_form_apartment_interest

## after form (there are multiple of these)
  > within_form_apartment_interest
  > back_from_within_form_apartment_interest
      - form{"name": null}
      - utter_thanks

or is there any reason why this wouldn’t work?

2reactions
tmbocommented, Aug 23, 2019

I don’t remember a specific reason why that is the case. the issue might be the following: The checkpoints in a case like this:

## within form interest blank
> within_form_apartment_interest
> back_from_within_form_apartment_interest

Are probably interpreted as start checkpoints. So the resulting block will have two start checkpoints and no end checkpoints (technically it will have an end checkpoint, the default one END).

This would mean that the back_from_within_form_apartment_interest checkpoint will just be ignored as a start checkpoint (since there is no other story that uses this as an end checkpoint, therefore this block can not be concatenated to something using this start checkpoint).

For the within_form_apartment_interest checkpoint this should mean that any block that ends with this checkpoint will reach the story end immediately. E.g. combining the above story block with

## beginning of form
* some_intent
      - form_apartment_interest
      - form{"name": "form_apartment_interest"}
  > within_form_apartment_interest

will lead to the following story:

## beginning of form
* some_intent
      - form_apartment_interest
      - form{"name": "form_apartment_interest"}
Read more comments on GitHub >

github_iconTop Results From Across the Web

Stories - Rasa
During training, Rasa does not call the action server. ... Unlike regular stories, checkpoints are not restricted to starting with user input.
Read more >
Use Early Stopping to Halt the Training of Neural Networks At ...
Early stopping is a method that allows you to specify an arbitrary large number of training epochs and stop training once the model...
Read more >
ray.tune.tune — Ray 2.2.0 - the Ray documentation
When a SIGINT signal is received (e.g. through Ctrl+C), the tuning run will gracefully shut down ... Ray will recover from the latest...
Read more >
How transform checkpoints work | Elasticsearch Guide [8.5]
Each time a transform examines the source indices and creates or updates the destination index, it generates a checkpoint. If your transform runs...
Read more >
Save only best weights with huggingface transformers
There may be better ways to avoid too many checkpoints and selecting the best model. So far you can not save only the...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found