cannot use safe JSON deserialization for Dialog State object

Version

4.5.1

Describe the bug

To save/load the dialog state, the bot need to serialize/deserialize the dialog states with JSON. Currently, the dialog states contains a list of dictionaries of objects, which is black box to the bot author programmer. To deserialize the string to the dialog state , the type information must be put into the JSON when searlizing the object ot string. For example, with Newtonsoft.Json, we need to specify TypeNameHandling = TypeNameHandling.All for serializer, just like the botbuilder sample code

But it is not safe to put the type information into the serialized JSON string, because an attacker could modify the serialized data to include unexpected types to inject objects with malicious side effects. There are some published document about it:

• Do not use insecure deserializer ObjectStateFormatter
• (MS Internal Security Requirement) Safe deserializers must be used in functions that process untrusted data

Per the above documents, when we serialize/deserialize the dialog state, we should specify specify TypeNameHandling = TypeNameHandling.All for JSON serializer. But currently it will cause exception in Bot Builder.

Bot Framework should provide a way to allow the bot to serialize/deserialize the dialog state safely.

To Reproduce

Steps to reproduce the behavior:

1. download the sample code of BotBuilder: 42.scaleout
2. change the code in the line from

private static readonly JsonSerializer StateJsonSerializer = new JsonSerializer() { TypeNameHandling = TypeNameHandling.All };

to

private static readonly JsonSerializer StateJsonSerializer = new JsonSerializer() { TypeNameHandling = TypeNameHandling.None };

1. Compile the project and run it
2. Use the bot emulator to talk to the bot

Actuall behavior

Exception happens:

Unable to cast object of type 'Newtonsoft.Json.Linq.JObject' to type 'Microsoft.Bot.Builder.Dialogs.DialogState'.

   at Microsoft.Bot.Builder.Dialogs.ComponentDialog.ContinueDialogAsync(DialogContext outerDc, CancellationToken cancellationToken) in d:\a\1\s\libraries\Microsoft.Bot.Builder.Dialogs\ComponentDialog.cs:line 75
   at Microsoft.Bot.Builder.Dialogs.DialogContext.ContinueDialogAsync(CancellationToken cancellationToken) in d:\a\1\s\libraries\Microsoft.Bot.Builder.Dialogs\DialogContext.cs:line 160
   at Microsoft.Bot.Builder.Dialogs.DialogExtensions.RunAsync(Dialog dialog, ITurnContext turnContext, IStatePropertyAccessor`1 accessor, CancellationToken cancellationToken) in d:\a\1\s\libraries\Microsoft.Bot.Builder.Dialogs\DialogExtensions.cs:line 18
   at Microsoft.BotBuilderSamples.DialogHost.RunTurnAsync(Dialog dialog, ITurnContext turnContext, JObject state, CancellationToken cancellationToken) in C:\src\others\BotBuilder-Samples\samples\csharp_dotnetcore\42.scaleout\DialogHost.cs:line 67

Expected behavior

There is no error.

Screenshots

[bug]

[kvbreddy]

I have fixed this issue for OAuthPrompt Dialog in this PR. May be someone can do a similar fix for other Dialogs as well.

[kvbreddy]

@EricDahlvang , Can this issue be reopened.

The above issue is still not resolved. DialogState serialization and deserialization works only if TypeNameHandling = TypeNameHandling.All. TypeNameHandling = TypeNameHandling.None still does not work, because in most places DialogInstance.State is used and directly casted to different types which requires $type to work.

For example, in our case : var promptState = (IDictionary<string, object>)state[PersistedState]; throws an exception when using TypeNameHandling = TypeNameHandling.None as the state[PersistedState] is a JObject.