Bug: OutOfMemory Exception When Creating the JSON Output

Describe the bug When running ApplicationInspector on a codebase that results in a big report, creating the JSON file results in an exception of type System.OutOfMemory.

To Reproduce

1. Run ApplicationInspector on a target that creates a large report with -v debug flag.
   • In my case I am running it on 600-700MBs of C/C++ code,
2. Notice the error message in the log file after it crashes in the end.

Error Messages

Command prompt

96% source files processedDecompressing files...
Decompressing files...
100% source files processed
Preparing report
A runtime error has occured.  Please see log file for more information.

From the log file with -v debug

2020-03-13T19:45:12 1 ERROR - Runtime error: Exception of type 'System.OutOfMemoryException' was thrown.    at System.Text.StringBuilder.ToString()
   at System.IO.StringWriter.ToString()
   at Newtonsoft.Json.JsonConvert.SerializeObjectInternal(Object value, Type type, JsonSerializer jsonSerializer)
   at Microsoft.AppInspector.LiquidWriter.WriteApp(AppProfile app)
   at Microsoft.AppInspector.AnalyzeCommand.FlushAll()
   at Microsoft.AppInspector.AnalyzeCommand.Run()
   at Microsoft.AppInspector.Program.RunAnalyzeCommand(AnalyzeCommandOptions opts)
   at Microsoft.AppInspector.Program.<>c.<Main>b__6_0(AnalyzeCommandOptions opts)
   at CommandLine.ParserResultExtensions.MapResult[T1,T2,T3,T4,T5,TResult](ParserResult`1 result, Func`2 parsedFunc1, Func`2 parsedFunc2, Func`2 parsedFunc3, Func`2 parsedFunc4, Func`2 parsedFunc5, Func`2 notParsedFunc)
   at Microsoft.AppInspector.Program.Main(String[] args)

Expected behavior Not crash. I have downloaded a lot of RAM.

Desktop (please complete the following information):

• OS: Windows 10
• Version 1903

[guyacosta]

Solution serializes directly to the file stream now for json writer. Note the utf8json while significantly better for performance requires use of the PrettyPrint method to format the output which only accepts a byte[] not an <T> object hence would require conversion to a string first anyway and not help. I did note the html output does actually require the AppProfile object to be converted to json for access in the AppInspector.js files for HTML UI so there is a conversion that is needed there and I’m unable to work around that. If you need HTML based output size has some memory limitations but json or text should work fine now.

[gfs]

@parsiya Per line 47 of LiquidWriter.cs the this crash is when the full JSON is being stuffed in the HTML output - it is not separately generating a JSON output, so I don’t think that a switch for #167 would do anything here.

@guyacosta worth investigating another JSON parsing library? I’ve been using https://github.com/neuecc/Utf8Json. Its not too complex to replace Newtonsoft.Json with it.