Frequent "file being used by another process IOException" when building documentation

Hi, I maintain https://github.com/microsoft/FactoryOrchestrator, which uses DefaultDocumentation to generate markdown files.

When building both locally and in ADO I frequently hit “Unhandled Exception: System.AggregateException: One or more errors occurred. —> System.Exception: Error while writing documentation for Microsoft.FactoryOrchestrator.Core —> System.IO.IOException: The process cannot access the file ‘D:\FactoryOrchestrator\bin\DefaultDocumentation\CoreLibrary\Microsoft-FactoryOrchestrator-Core.md’ because it is being used by another process.”

By adding basic retry logic I am able to work around the issue, but I couldn’t figure out the root cause of the exception, all threads are writing different files at the time of the exception.

A dump captured from running the debug version of your tool (0.6.13) I built locally is at https://1drv.ms/u/s!AigADXVMXV_6t8ML930SnhDu6vxfog?e=xOd4uQ

You might also be able to repro by checking out and building https://github.com/microsoft/FactoryOrchestrator

The retry logic I added (EDIT, one retry works fine in debug, but not for release, updated to 10 retries):

        public void WriteDocumentation(string outputFolderPath, int retryIOException = 0)
        {
            _docItems.Values.Where(i => i.GeneratePage).AsParallel().ForAll(i =>
            {
                try
                {
                    using DocumentationWriter writer = new DocumentationWriter(_fileNameMode, _nestedTypeVisibility, _wikiLinks, _docItems, _links, outputFolderPath, i);
                    i.WriteDocumentation(writer);
                    
                }
                catch (Exception exception)
                {
                    if (retryIOException < 10 && exception is System.IO.IOException)
                    {
                        // Retry up to 10 times on IOException
                        WriteDocumentation(outputFolderPath, ++retryIOException);
                    }
                    else
                    {
                        throw new Exception($"Error while writing documentation for {i.FullName}", exception);
                    }
                }
            });
        }

While I would love to root cause this and get a real fix, maybe adding this logic or something similar in the interm might help me and any others who hit this?

Removing AsParallel().ForAll and using a serial foreach also resolves the issue.

[spaceisfun]

I really appreciate you addressing this so quickly @Doraku !

Thanks for writing such a useful piece of OSS, this was the best doc generator out of the (admittedly only a handful of) ones I tried 😃, and your prompt resolution here only helps confirm that 🥇 .

[Doraku]

I can’t reproduce it but I have a fairly old cpu, so maybe there’s something wrong with the file system when trying to write too many files at the same time 😕 I took your word for it and removed the parallel execution, premature optimization probably biting me in the ass eh. As you said it didn’t do much 😃