EntityMasker component

What problem are you trying to solve?

Currently, the intent classifier doesn’t leverage the information extracted by the entity extractors. In general, entity values add noise to the training data of the intent classifier, because they vary a lot. It would be nice if entity values could be normalized to placeholders instead (that is, entity values are “masked” to the intent classifier, so it doesn’t see them but sees placeholders insted).

What’s your suggested solution?

Add a EntityMasker component that preprocesses the intent classifier input by substituting entity values with placeholders during training and inference.

Examples (if relevant)

Imagine an intent classifier that must detect the intent buy with the following examples:

• buy an apple
• buy a chair
• buy me a ticket

It would be a lot easier for the intent classifier to detect the intent if the entities are masked with placeholders, like this:

• buy an item
• buy a item
• buy me a item

This way, it will be a lot easier for the intent classifier to learn the intent buy, because the noise added by the entity values is removed. The only information that remains is the fact that there is an entity of type item, but the actual values are masked.

Is anything blocking this from being implemented? (if relevant)

No response

Definition of Done

Add the EntityMasker component.

[koaning]

Exalate commented:

koaning commented:

@joejuzl figured I’d ping you on this one, there may be a nice use-case for the graphical backend here. This component would need to sit between an entity extractor and DIET.

@Maxinho96 while your use-case here is valid, one thing to keep in mind: since DIET is able to detect both intents and entities they can also influence each other. This is actually by design! <g-emoji class="g-emoji" alias="smile" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/1f604.png"> 😄 </g-emoji> Sometimes we’d want the entities to influence the intent prediction! The presence of an item might exclude some intents and that has predictive power.

[sync-by-unito[bot]]

➤ Maxime Verger commented:

💡 Heads up! We’re moving issues to Jira: https://rasa-open-source.atlassian.net/browse/OSS.

From now on, this Jira board is the place where you can browse (without an account) and create issues (you’ll need a free Jira account for that). This GitHub issue has already been migrated to Jira and will be closed on January 9th, 2023. Do not forget to subscribe to the corresponding Jira issue!

➡️ More information in the forum: https://forum.rasa.com/t/migration-of-rasa-oss-issues-to-jira/56569.