CloudBlockBlob.DownloadText() handles UTF8 BOM incorrectly

CloudBlockBlob.DownloadText() behaves differently than File.ReadAllText in respect to UTF8 pre-amble/BOM

Repro: Create an XML File in Visual Studio and upload it to a Cloud Blob container. The file will begin with a BOM (EF BB BF). Then download it using CloudBlockBlob.DownloadText() and pass the resulting string to XDocument.Parse. The parser will fail with XMLException - “Data at the root level is invalid. Line 1, position 1.”. Failing code:

var storageAccount = CloudStorageAccount.Parse("connectionString");
var blobClient = storageAccount.CreateCloudBlobClient();
var container = blobClient.GetContainerReference("MyContainer");
var blob = container.GetBlockBlobReference("my.xml");
var s = blob.DownloadText();
var x = XDocument.Parse(s);

A workaround suggested at http://stackoverflow.com/questions/2111586/parsing-xml-string-to-an-xml-document-fails-if-the-string-begins-with-xml by Dave Cluderay suggests passing the read string through StreamReader. Working code

        var storageAccount = CloudStorageAccount.Parse("connectionString");
        var blobClient = storageAccount.CreateCloudBlobClient();
        var container = blobClient.GetContainerReference("MyContainer");
        var blob = container.GetBlockBlobReference("my.xml");
        var s = blob.DownloadText();
        using (var memoryStream = new MemoryStream(Encoding.UTF8.GetBytes(s)))
        {
            using (var streamReader = new StreamReader(memoryStream))
            {
                var x = XDocument.Load(streamReader);
            }
        }

Moved from https://github.com/Azure/azure-sdk-for-net/issues/626#

[guardrex]

@joefeser 👍 I’m way late to the party, but …

We could detect and strip out the BOM ourselves, but this is a potentially surprising transformation of the blob’s original data

It would have been nice though to have an overload .DownloadText(bool stripByteOrderMark), so that the original method would just do the un-“surprising” thing and the overload would allow the dev to explicitly call for stripping the BOM.

[r-aghaei]

I created an extension method DownloadString for CloudBlockBlob and used the same code which is used in WebClient.DownloadString. Here you can find the code: How to get rid of BOM when downloading text from azure blob