TestFilterBuilder builds invalid XML

(Updated)

This implementation mishandles surrogate pairs. Control characters other than tab, carriage return and line feed cannot be represented in XML and should throw, IIRC. (See https://github.com/nunit/nunit3-vs-adapter/issues/484.)

https://github.com/nunit/nunit-console/blob/af89a437153955fb3aea0ea168965f596a6f3066/src/NUnitEngine/nunit.engine/Services/TestFilterBuilder.cs#L82-L90

GetFilter() should be using XmlWriter.Create to a StringWriter. This will not only get us out of the responsibility of implementing the XML spec, it will probably also result in some significant performance gains. (Judging by the chained string.Replace calls.)

(Test names containing control characters cannot be represented in XML without first encoding using base64 or something custom. This will need to be dealt separately with in the framework.)

@nunit/engine-team Does this seem right to you? Can we let @MatthewBeardmore get started on this if he is interested?

[ChrisMaddock]

There’s a lot of moving parts in this issue, but I think Joseph has summed it up quite well here: https://github.com/nunit/nunit/issues/3063#issuecomment-432788447

Currently, for the example shown at https://github.com/nunit/nunit3-vs-adapter/issues/484, the framework by default gives that a test name which can’t legally be represented by XML. This seems to cause unpredictability when the framework/engine/runners all rely on xml to encode their communications.

IMO, the framework should be changed to not create xml-invalid names by default, and prevent users overriding this with SetName. For the engine’s part, it should only:

1. Expect that all messages received from framework(s) are valid XML.
2. Prevent runners creating content for the framework which cannot be encoding in XML. The only example I can think of here is a runner attempting to create a test filter which can’t be encoded in XML, as happened in this situation.

If the framework doesn’t create test names as per item 1, then I can’t think of any valid need for item 2 - the engine just needs to handle this elegantly.

[CharliePoole]

@jnm2 I may be missing something but here’s what I was thinking…

There is no valid method name containing control characters. Therefore, no default test name can contain control characters and such a test name can only be created by the user using SetName. However, such a test name serves no useful purpose if we can’t use it for filtering so why not just eliminate such names by disallowing control characters in SetName?

This might be “breaking” in the sense that you would get an error message rather than just fail to select the test, but so what?