Core Assistant: Cannot disambiguate similar intents in the same LUIS recognizer

Describe the bug

Basic Assistant template has a duplicated intents recognized capability which I’m unable to trigger. Some of this is related to microsoft/BotFramework-Composer#7404 but after inspecting trace it would appear that IncludeAllIntents isn’t being set on the call to LUIS meaning your only going to get the topIntent back not the other intents and scores meaning this feature cannot work when LUIS is being used.

{
  "activity": {
    "type": "trace",
    "timestamp": "2021-04-27T14:12:01.477Z",
    "serviceUrl": "http://localhost:5000",
    "channelId": "emulator",
    "from": {
      "id": "ec66a842-b822-4078-92c4-d2bf4890942d",
      "name": "Bot",
      "role": "bot"
    },
    "conversation": {
      "id": "04b1056c-dc37-44ad-8764-ea2ff77add8d|livechat"
    },
    "recipient": {
      "id": "91b9c1ca-4273-4517-87a4-6cf4ffe6a8ab",
      "role": "user"
    },
    "locale": "en-us",
    "replyToId": "1d2ffde0-5652-4b7a-a5d9-3c7f8702e5d0",
    "label": "LuisV3 Trace",
    "valueType": "https://www.luis.ai/schemas/trace",
    "value": {
      "recognizerResult": {
        "text": "book",
        "alteredText": null,
        "intents": {
          "None": {
            "score": 0.79196495
          }
        },
        "entities": {}
      },
      "luisModel": {
        "ModelID": "44388ba7-de06-46e6-9c98-090b6219f5e9"
      },
      "luisOptions": {
        "IncludeAllIntents": false,
        "IncludeInstanceData": false,
        "IncludeAPIResults": false,
        "Log": true,
        "DynamicLists": null,
        "ExternalEntities": null,
        "PreferExternalEntities": true,
        "DateTimeReference": null,
        "Slot": "production",
        "Version": null
      },
      "luisResult": {
        "query": "book",
        "prediction": {
          "topIntent": "None",
          "intents": {
            "None": {
              "score": 0.79196495
            }
          },
          "entities": {}
        }
      }
    },
    "name": "LuisRecognizer",
    "id": "bfc80184-d739-470e-9d93-dbd10d6cabfa",
    "localTimestamp": "2021-04-27T15:12:01+01:00"
  },
  "id": "0f59b4bf-598d-4d26-9621-11fac3acbffc",
  "timestamp": 1619532721477
}

Version

2.0.0-nightly-239644

Browser

• Electron distribution
• Chrome
• Safari
• Firefox
• Edge

OS

• macOS
• Windows
• Ubuntu

To Reproduce

Steps to reproduce the behavior:

1. Create Basic Assistant
2. Create two dialogs with overlapping intents (be aware of microsoft/BotFramework-Composer#7404)
3. Observe LUIS trace in Composer and see that only topIntent is being returned

Expected behavior

Disambiguation to work!

[tomlm]

Tai is correct that this event is fired currently only with Cross trained QnA/LUIS models.

I do have an ThresholdRecognizer implementation in iciclecreek.bot/ThresholdRecognizer.cs at master · tomlm/iciclecreek.bot (github.com) which applies a threshold to any recognizer.

In this package

This example will fire OnChooseIntent for intents that are within .1 of each other from the inner recognizer.

{
    "recognizer": {
        "$kind":"Iciclecreek.ThresholdRecognizer",
        "threshold": 0.1, 
        "recognizer": { 
            ...luis recognizer with all intents returned...
         }
    }
    ...
} 

[taicchoumsft]

We have two major options here:

1. High Cost, High Risk (but right thing to do given current design): Copy OrchestratorRecognizer’s disambiguation logic to LuisRecognizer.

   This is done at SDK level - must make changes in Node and C# SDK, also comes with LUIS Recognizer schema changes. No Composer changes necessary, no need to enable LUIS’s includeAllIntents.

2. Low Cost, Medium Risk: Shift the disambiguation logic from SDK into Composer.

   Mimic SDK disambiguation logic inside Composer’s OnIntent trigger template. If there are ambiguous intents, then the OnIntent trigger calls the disambiguation trigger. This means we add a lot of template code (visible to Composer user) to the OnIntent trigger.