Some tests depend on machine specifics

.NET Core Version: 5.0 master branch

Have you experienced this same bug with .NET Framework?: not applicable

Problem description: After removing culture-specific test failures (tracked by #2734) there are a few more tests failing for me which I believe happen due to relying on machine specific conditions.

list of failed tests

* System.Windows.Forms.Tests.CursorTests.Cursor_Clip_Set_GetReturnsExpected
Assert.Equal() Failure
Expected: {X=0,Y=0,Width=3456,Height=1080}
Actual:   {X=0,Y=0,Width=3840,Height=1080}
   at System.Windows.Forms.Tests.CursorTests.Cursor_Clip_Set_GetReturnsExpected() in D:\projects\winforms\src\System.Windows.Forms\tests\UnitTests\System\Windows\Forms\CursorTests.cs:line 176

* System.Windows.Forms.Tests.ListViewTests.ListView_GetItemRect_InvokeWithHandle_ReturnsExpected
Assert.True() Failure
Expected: True
Actual:   False
   at System.Windows.Forms.Tests.ListViewTests.ListView_GetItemRect_InvokeWithHandle_ReturnsExpected() in D:\projects\winforms\src\System.Windows.Forms\tests\UnitTests\System\Windows\Forms\ListViewTests.cs:line 3680

* System.Windows.Forms.Tests.ListViewTests.ListView_GetItemRect_InvokeWithoutHandle_ReturnsExpectedAndCreatedHandle
Assert.True() Failure
Expected: True
Actual:   False
   at System.Windows.Forms.Tests.ListViewTests.ListView_GetItemRect_InvokeWithoutHandle_ReturnsExpectedAndCreatedHandle() in D:\projects\winforms\src\System.Windows.Forms\tests\UnitTests\System\Windows\Forms\ListViewTests.cs:line 3645

* System.Windows.Forms.Tests.MonthCalendarTests.MonthCalendar_CalendarDimensions_SetAreaOfTwelve_GetReturnsExpected(width: 12, height: 1)
Assert.True() Failure
Expected: True
Actual:   False
   at System.Windows.Forms.Tests.MonthCalendarTests.MonthCalendar_CalendarDimensions_SetAreaOfTwelve_GetReturnsExpected(Int32 width, Int32 height) in D:\projects\winforms\src\System.Windows.Forms\tests\UnitTests\System\Windows\Forms\MonthCalendarTests.cs:line 586

* System.Windows.Forms.Tests.MonthCalendarTests.MonthCalendar_CalendarDimensions_SetWithHandleAreaOfTwelve_GetReturnsExpected(width: 12, height: 1)
Assert.True() Failure
Expected: True
Actual:   False
   at System.Windows.Forms.Tests.MonthCalendarTests.MonthCalendar_CalendarDimensions_SetWithHandleAreaOfTwelve_GetReturnsExpected(Int32 width, Int32 height) in D:\projects\winforms\src\System.Windows.Forms\tests\UnitTests\System\Windows\Forms\MonthCalendarTests.cs:line 644

* System.Windows.Forms.Tests.MonthCalendarTests.MonthCalendar_SingleMonthSize_GetWithHandle_ReturnsExpected
Assert.True() Failure
Expected: True
Actual:   False
   at System.Windows.Forms.Tests.MonthCalendarTests.MonthCalendar_SingleMonthSize_GetWithHandle_ReturnsExpected() in D:\projects\winforms\src\System.Windows.Forms\tests\UnitTests\System\Windows\Forms\MonthCalendarTests.cs:line 2338

* System.Windows.Forms.Tests.ToolStripDropDownTests.ToolStripDropDown_Ctor_Default
Assert.Equal() Failure
Expected: 2
Actual:   79
   at System.Windows.Forms.Tests.ToolStripDropDownTests.ToolStripDropDown_Ctor_Default() in D:\projects\winforms\src\System.Windows.Forms\tests\UnitTests\System\Windows\Forms\ToolStripDropDownTests.cs:line 142

* System.Windows.Forms.Tests.ToolStripDropDownTests.ToolStripDropDown_Location_Set_GetReturnsExpected(value: {X=-1,Y=-2}, expected: {X=0,Y=0}, expectedLocationChangedCallCount: 0)
Assert.Equal() Failure
Expected: 0
Actual:   77
   at System.Windows.Forms.Tests.ToolStripDropDownTests.ToolStripDropDown_Location_Set_GetReturnsExpected(Point value, Point expected, Int32 expectedLocationChangedCallCount) in D:\projects\winforms\src\System.Windows.Forms\tests\UnitTests\System\Windows\Forms\ToolStripDropDownTests.cs:line 1966

* System.Windows.Forms.Tests.ToolStripDropDownTests.ToolStripDropDown_Location_Set_GetReturnsExpected(value: {X=0,Y=2}, expected: {X=0,Y=2}, expectedLocationChangedCallCount: 1)
Assert.Equal() Failure
Expected: 0
Actual:   77
   at System.Windows.Forms.Tests.ToolStripDropDownTests.ToolStripDropDown_Location_Set_GetReturnsExpected(Point value, Point expected, Int32 expectedLocationChangedCallCount) in D:\projects\winforms\src\System.Windows.Forms\tests\UnitTests\System\Windows\Forms\ToolStripDropDownTests.cs:line 1966

* System.Windows.Forms.Tests.ToolStripDropDownTests.ToolStripDropDown_Location_Set_GetReturnsExpected(value: {X=1,Y=0}, expected: {X=1,Y=0}, expectedLocationChangedCallCount: 1)
Assert.Equal() Failure
Expected: 1
Actual:   77
   at System.Windows.Forms.Tests.ToolStripDropDownTests.ToolStripDropDown_Location_Set_GetReturnsExpected(Point value, Point expected, Int32 expectedLocationChangedCallCount) in D:\projects\winforms\src\System.Windows.Forms\tests\UnitTests\System\Windows\Forms\ToolStripDropDownTests.cs:line 1966

* System.Windows.Forms.Tests.ToolStripDropDownTests.ToolStripDropDown_Location_Set_GetReturnsExpected(value: {X=1,Y=2}, expected: {X=1,Y=2}, expectedLocationChangedCallCount: 1)
Assert.Equal() Failure
Expected: 1
Actual:   77
   at System.Windows.Forms.Tests.ToolStripDropDownTests.ToolStripDropDown_Location_Set_GetReturnsExpected(Point value, Point expected, Int32 expectedLocationChangedCallCount) in D:\projects\winforms\src\System.Windows.Forms\tests\UnitTests\System\Windows\Forms\ToolStripDropDownTests.cs:line 1966

There is a compromise to make between developer convenience and how exactly you require tests matching their OS environment, if they are too strict developers can run tests only in a VM.

So far I could identify:

• Some MonthCalendar tests generate a huge control and rely on the OS cutting them off. If you actually have a big work area (e.g. two screens) this leads to test failures since the controls are no longer cut off and are larger than the test expects.
• Some tests mix up coordinate spaces and do not consider the fact that the taskbar is not necessarily at the bottom. When the OS ensures the taskbar is not covered some tests fail because the forms/controls are not at the location they expect them to be. This is easily worked around by considering the work area inside the test, not the virtual screen space.

I’ll put together a PR, so you can then decide which of the changes are reasonable enough to take.

Expected behavior: Tests are (within reason) coded to be independent of machine characteristics.

Minimal repro: Having either of these characteristics will fail some tests:

• Set the taskbar to the left side of the monitor (instead of default bottom) and run the tests
• Use a multi-screen machine and run the tests (very wide or very high work area)

[RussKie]

Created a net472 app, and it starts looking like a bug in .NET runtime… 🤯

I created a standalone multi-targeting console app and ran it testing different start days (Sat and Wed):

PS C:\Development\!Tests\ConsoleApp-CultureInfo> dotnet run -f net472
Running: .NET Framework 4.8.4180.0
CultureInfo.CurrentCulture.DateTimeFormat.FirstDayOfWeek: Saturday

PS C:\Development\!Tests\ConsoleApp-CultureInfo> dotnet run -f net48
Running: .NET Framework 4.8.4180.0
CultureInfo.CurrentCulture.DateTimeFormat.FirstDayOfWeek: Saturday

PS C:\Development\!Tests\ConsoleApp-CultureInfo> dotnet run -f netcoreapp3.1
Running: .NET Core 3.1.2
CultureInfo.CurrentCulture.DateTimeFormat.FirstDayOfWeek: Saturday

PS C:\Development\!Tests\ConsoleApp-CultureInfo> dotnet run -f net5.0
Running: .NET 5.0.0-preview.4.20213.10
CultureInfo.CurrentCulture.DateTimeFormat.FirstDayOfWeek: Saturday

PS C:\Development\!Tests\ConsoleApp-CultureInfo> dotnet run -f net472
Running: .NET Framework 4.8.4180.0
CultureInfo.CurrentCulture.DateTimeFormat.FirstDayOfWeek: Wednesday

PS C:\Development\!Tests\ConsoleApp-CultureInfo> dotnet run -f net48
Running: .NET Framework 4.8.4180.0
CultureInfo.CurrentCulture.DateTimeFormat.FirstDayOfWeek: Wednesday

PS C:\Development\!Tests\ConsoleApp-CultureInfo> dotnet run -f netcoreapp3.1
Running: .NET Core 3.1.2
CultureInfo.CurrentCulture.DateTimeFormat.FirstDayOfWeek: Wednesday

PS C:\Development\!Tests\ConsoleApp-CultureInfo> dotnet run -f net5.0
Running: .NET 5.0.0-preview.4.20213.10
CultureInfo.CurrentCulture.DateTimeFormat.FirstDayOfWeek: Wednesday

Then I pointed to winforms/.dotnet and… 💥

PS C:\Development\!Tests\ConsoleApp-CultureInfo> $env:PATH="C:\Development\winforms\.dotnet;$env:PATH"
PS C:\Development\!Tests\ConsoleApp-CultureInfo> powershell
PS C:\Development\!Tests\ConsoleApp-CultureInfo> dotnet run -f net5.0
Running: .NET 5.0.0-preview.6.20264.1
CultureInfo.CurrentCulture.DateTimeFormat.FirstDayOfWeek: Sunday

[RussKie]

I think this issue has resolved. I’m no longer getting any locale specific failures. Holler if you still do, and we can reopen it.