e2e error when using "alias"

I’m checking if the alias mechanic works on the main branch. It seemed to work just fine for DIET so I figured adding it to TED.

I added an e2e example.

version: "2.0"

stories:

- story: happy path
  steps:
  - intent: greet
  - action: utter_greet
  - intent: mood_great
  - action: utter_happy

- story: sad path 1
  steps:
  - intent: greet
  - action: utter_greet
  - intent: mood_unhappy
  - action: utter_cheer_up
  - action: utter_did_that_help
  - user: "such very indeed yes mjas"
  - action: utter_happy

- story: sad path 2
  steps:
  - intent: greet
  - action: utter_greet
  - intent: mood_unhappy
  - action: utter_cheer_up
  - action: utter_did_that_help
  - intent: deny
  - action: utter_goodbye

I configured TED to only listen to one of the countvectorisers.

recipe: default.v1

language: en

pipeline:
  - name: WhitespaceTokenizer
  - name: RegexFeaturizer
  - name: LexicalSyntacticFeaturizer
  - name: CountVectorsFeaturizer
    alias: feats1
  - name: CountVectorsFeaturizer
    alias: feats2
    analyzer: char_wb
    min_ngram: 1
    max_ngram: 4
  - name: DIETClassifier
    featurizers:
      - feats1
      - feats2
    epochs: 50
    constrain_similarities: true
  - name: EntitySynonymMapper
  - name: ResponseSelector
    epochs: 50
    constrain_similarities: true
  - name: FallbackClassifier
    threshold: 0.3
    ambiguity_threshold: 0.1
  - name: RegexEntityExtractor
    use_lookup_tables: true

policies:
  - name: MemoizationPolicy
  - name: RulePolicy
  - name: UnexpecTEDIntentPolicy
    max_history: 5
    epochs: 50
  - name: TEDPolicy
    featurizers:
      - feats1
    max_history: 5
    epochs: 50
    constrain_similarities: true

This resulted in an error.

> rasa train
2021-10-19 13:08:13 WARNING  rasa.shared.utils.common  - The end-to-end training is currently experimental and might change or be removed in the future 🔬 Please share your feedback on it in the forum (https://forum.rasa.com) to help us make this feature ready for production.
Processed story blocks: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 195.92it/s, # trackers=1]
Processed story blocks: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 63.30it/s, # trackers=3]
Processed story blocks: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 116.48it/s, # trackers=12]
Processed story blocks: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 23.67it/s, # trackers=39]
Processed rules: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 1278.56it/s, # trackers=1]
2021-10-19 13:08:15 INFO     rasa.engine.training.hooks  - Starting to train component 'RegexFeaturizerGraphComponent'.
2021-10-19 13:08:15 INFO     rasa.engine.training.hooks  - Finished training component 'RegexFeaturizerGraphComponent'.
2021-10-19 13:08:15 INFO     rasa.engine.training.hooks  - Starting to train component 'MemoizationPolicyGraphComponent'.
Processed trackers: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 845.91it/s, # action=12]
Processed actions: 12it [00:00, 6995.36it/s, # examples=12]
2021-10-19 13:08:15 INFO     rasa.engine.training.hooks  - Finished training component 'MemoizationPolicyGraphComponent'.
2021-10-19 13:08:15 INFO     rasa.engine.training.hooks  - Starting to train component 'RulePolicyGraphComponent'.
Processed trackers: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 1278.36it/s, # action=5]
Processed actions: 5it [00:00, 8487.06it/s, # examples=4]
Processed trackers: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 824.35it/s, # action=12]
Processed trackers: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 1122.52it/s]
Processed trackers: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 836.15it/s]
2021-10-19 13:08:16 INFO     rasa.engine.training.hooks  - Finished training component 'RulePolicyGraphComponent'.
2021-10-19 13:08:16 INFO     rasa.engine.training.hooks  - Starting to train component 'LexicalSyntacticFeaturizerGraphComponent'.
2021-10-19 13:08:16 INFO     rasa.engine.training.hooks  - Finished training component 'LexicalSyntacticFeaturizerGraphComponent'.
2021-10-19 13:08:17 INFO     rasa.engine.training.hooks  - Starting to train component 'CountVectorsFeaturizerGraphComponent'.
2021-10-19 13:08:17 INFO     rasa.nlu.featurizers.sparse_featurizer.count_vectors_featurizer  - 82 vocabulary items were created for text attribute.
2021-10-19 13:08:17 INFO     rasa.engine.training.hooks  - Finished training component 'CountVectorsFeaturizerGraphComponent'.
2021-10-19 13:08:17 INFO     rasa.engine.training.hooks  - Starting to train component 'CountVectorsFeaturizerGraphComponent'.
2021-10-19 13:08:17 INFO     rasa.nlu.featurizers.sparse_featurizer.count_vectors_featurizer  - 715 vocabulary items were created for text attribute.
2021-10-19 13:08:17 INFO     rasa.engine.training.hooks  - Finished training component 'CountVectorsFeaturizerGraphComponent'.
2021-10-19 13:08:18 INFO     rasa.engine.training.hooks  - Starting to train component 'DIETClassifierGraphComponent'.
Epochs: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████| 50/50 [00:14<00:00,  3.54it/s, t_loss=1.47, i_acc=1]
2021-10-19 13:08:32 INFO     rasa.engine.training.hooks  - Finished training component 'DIETClassifierGraphComponent'.
2021-10-19 13:08:33 INFO     rasa.engine.training.hooks  - Starting to train component 'EntitySynonymMapperGraphComponent'.
2021-10-19 13:08:33 INFO     rasa.engine.training.hooks  - Finished training component 'EntitySynonymMapperGraphComponent'.
2021-10-19 13:08:33 INFO     rasa.engine.training.hooks  - Starting to train component 'RegexEntityExtractorGraphComponent'.
/workspace/.pip-modules/lib/python3.7/site-packages/rasa/shared/utils/io.py:99: UserWarning: No lookup tables or regexes defined in the training data that have a name equal to any entity in the training data. In order for this component to work you need to define valid lookup tables or regexes in the training data.
2021-10-19 13:08:33 INFO     rasa.engine.training.hooks  - Finished training component 'RegexEntityExtractorGraphComponent'.
2021-10-19 13:08:33 INFO     rasa.engine.training.hooks  - Starting to train component 'ResponseSelectorGraphComponent'.
2021-10-19 13:08:33 INFO     rasa.nlu.selectors.response_selector  - Retrieval intent parameter was left to its default value. This response selector will be trained on training examples combining all retrieval intents.
2021-10-19 13:08:33 INFO     rasa.engine.training.hooks  - Finished training component 'ResponseSelectorGraphComponent'.
2021-10-19 13:08:33 INFO     rasa.engine.training.hooks  - Starting to train component 'TEDPolicyGraphComponent'.
Processed trackers: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████| 120/120 [00:00<00:00, 1401.07it/s, # action=30]
Traceback (most recent call last):
  File "/workspace/.pip-modules/lib/python3.7/site-packages/rasa/engine/graph.py", line 459, in __call__
    output = self._fn(self._component, **run_kwargs)
  File "/workspace/.pip-modules/lib/python3.7/site-packages/rasa/core/policies/ted_policy.py", line 713, in train
    self.run_training(model_data, label_ids)
  File "/workspace/.pip-modules/lib/python3.7/site-packages/rasa/core/policies/ted_policy.py", line 646, in run_training
    self._entity_tag_specs,
  File "/workspace/.pip-modules/lib/python3.7/site-packages/rasa/core/policies/ted_policy.py", line 1191, in __init__
    self._prepare_layers()
  File "/workspace/.pip-modules/lib/python3.7/site-packages/rasa/core/policies/ted_policy.py", line 1218, in _prepare_layers
    name, self.data_signature[name], is_label_attribute=False
  File "/workspace/.pip-modules/lib/python3.7/site-packages/rasa/core/policies/ted_policy.py", line 1273, in _prepare_input_layers
    attribute_name, attribute_signature, config_to_use
  File "/workspace/.pip-modules/lib/python3.7/site-packages/rasa/utils/tensorflow/rasa_layers.py", line 761, in __init__
    "The attribute signature must contain some sequence-level feature"
rasa.utils.tensorflow.exceptions.TFLayerConfigException: The attribute signature must contain some sequence-level featuresignatures but none were found.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/.pip-modules/bin/rasa", line 8, in <module>
    sys.exit(main())
  File "/workspace/.pip-modules/lib/python3.7/site-packages/rasa/__main__.py", line 117, in main
    cmdline_arguments.func(cmdline_arguments)
  File "/workspace/.pip-modules/lib/python3.7/site-packages/rasa/cli/train.py", line 59, in <lambda>
    train_parser.set_defaults(func=lambda args: run_training(args, can_exit=True))
  File "/workspace/.pip-modules/lib/python3.7/site-packages/rasa/cli/train.py", line 103, in run_training
    finetuning_epoch_fraction=args.epoch_fraction,
  File "/workspace/.pip-modules/lib/python3.7/site-packages/rasa/api.py", line 117, in train
    finetuning_epoch_fraction=finetuning_epoch_fraction,
  File "/workspace/.pip-modules/lib/python3.7/site-packages/rasa/model_training.py", line 179, in train
    **(nlu_additional_arguments or {}),
  File "/workspace/.pip-modules/lib/python3.7/site-packages/rasa/model_training.py", line 240, in _train_graph
    is_finetuning=is_finetuning,
  File "/workspace/.pip-modules/lib/python3.7/site-packages/rasa/engine/training/graph_trainer.py", line 108, in train
    graph_runner.run(inputs={PLACEHOLDER_IMPORTER: importer})
  File "/workspace/.pip-modules/lib/python3.7/site-packages/rasa/engine/runner/dask.py", line 106, in run
    dask_result = dask.get(run_graph, run_targets)
  File "/workspace/.pip-modules/lib/python3.7/site-packages/dask/local.py", line 565, in get_sync
    **kwargs
  File "/workspace/.pip-modules/lib/python3.7/site-packages/dask/local.py", line 503, in get_async
    for key, res_info, failed in queue_get(queue).result():
  File "/usr/local/lib/python3.7/concurrent/futures/_base.py", line 428, in result
    return self.__get_result()
  File "/usr/local/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File "/workspace/.pip-modules/lib/python3.7/site-packages/dask/local.py", line 545, in submit
    fut.set_result(fn(*args, **kwargs))
  File "/workspace/.pip-modules/lib/python3.7/site-packages/dask/local.py", line 237, in batch_execute_tasks
    return [execute_task(*a) for a in it]
  File "/workspace/.pip-modules/lib/python3.7/site-packages/dask/local.py", line 237, in <listcomp>
    return [execute_task(*a) for a in it]
  File "/workspace/.pip-modules/lib/python3.7/site-packages/dask/local.py", line 228, in execute_task
    result = pack_exception(e, dumps)
  File "/workspace/.pip-modules/lib/python3.7/site-packages/dask/local.py", line 223, in execute_task
    result = _execute_task(task, data)
  File "/workspace/.pip-modules/lib/python3.7/site-packages/dask/core.py", line 121, in _execute_task
    return func(*(_execute_task(a, cache) for a in args))
  File "/workspace/.pip-modules/lib/python3.7/site-packages/rasa/engine/graph.py", line 467, in __call__
    ) from e
rasa.engine.exceptions.GraphComponentException: Error running graph component for node train_TEDPolicy3.

This was a bit confusing. This element suggests that the CountVectorizer does not add sequence-level features, but it does … right?

The attribute signature must contain some sequence-level feature signatures but none were found.

[koaning]

The reason I brought it up was that I was of the impression that the research team is interested in the graph feature. The reasoning is that it should make it much easier to control what features are passed to TED, potentially allowing us to research speedups. I haven’t checked if it currently works in 2.8, this was an assumption on my part.

[koaning]

I think the issue is important but it’s a bit of a nichy use-case. I can imagine it’s only relevant to advanced users who use end-to-end, which is likely a minority of our users.

My impression is that there’s a push for end-to-end features after 3.0 is released and it may be wise to have this issue be part of that effort instead of the current 3.0 effort.

Adding research label now.