Can generate the file without BOM?

Steps to reproduce

dotnet new xxx

Expected behavior

Generate the all file without BOM

Actual behavior

Generate the all file with BOM

Environment data

dotnet --info output:

Product Information: Version: 2.1.104 Commit SHA-1 hash: 48ec687460

Runtime Environment: OS Name: Windows OS Version: 10.0.16299 OS Platform: Windows RID: win10-x64 Base Path: C:\Program Files\dotnet\sdk\2.1.104\

Microsoft .NET Core Shared Framework Host

Version : 2.0.6 Build : 74b1c703813c8910df5b96f304b0f2b78cdf194d

[aolszowka]

FWIW at least for C# to quote from the ECMA-334 5th Edition (https://www.ecma-international.org/publications/files/ECMA-ST/ECMA-334.pdf):

1. Conformance (PDF Page 25):

A conforming implementation of C# shall interpret characters in conformance with
the Unicode Standard. Conforming implementations shall accept Unicode source
files encoded with the UTF-8 encoding form.

7.1 Programs (PDF Page 35):

Conforming implementations shall accept Unicode source files encoded with the
UTF-8 encoding form (as defined by the Unicode standard), and transform them
into a sequence of Unicode characters. Implementations can choose to accept and
transform additional character encoding schemes (such as UTF-16, UTF-32, or
non-Unicode character mappings).

Nothing in here says that it has to contain the BOM, so if you are looking for the end all be all it will not be found in the standard…

That being said every Visual Studio version we have ever used the templates have always contained the BOM. We have commit hooks that enforce it for us internally due to some of the issues @sharwell as mentioned. For us there was a portion of code that contained some exotic characters required by a third party library that was garbled by text editors not properly respecting the fact that the file was indeed UTF-8. As he says having the BOM avoids more issues than it causes. YMMV.

[sharwell]

❗️ Source files need to be generated with BOM. Otherwise, certain editors will treat them in non-uniform manner and eventually someone will accidentally save the file with question marks (encoding error fallback character). Normally I see this in author names in files getting messed up, but recently we found a curly quote in dotnet/winforms which was incorrectly saved. These errors are easy to miss and (in many cases) hard to fix, so we create the file with BOM to avoid it altogether.

We have a secondary benefit that the BOM triggers an early exit in the automatic encoding detection algorithm in .NET, so editors like Visual Studio load files faster. It’s a small win and not really significant compared to the problem above, but I find it interesting. 😄