StringSpec test cases with multiline names broken

Bugs

I think I have found two bugs with StringSpecs. I’m unsure whether other testing styles are affected as well.

Environment

Here’s my environment. I’m unsure whether other environments are affected.

• Kotlin/JVM 1.3.72
• Kotest 4.0.5
• Gradle 6.4

Bug 1: Test Failures

Consider this psuedocode.

class TestSuite : StringSpec({
    """
    test
    case
    1
    """ {
        // test
    }

    """
    test
    case
    2
    """ {
        // test
    }

    "test case 3" {
        // test 
    }
})

If I run the entire suite, and the first two test cases fail, then the error message used for one of the cases is used as the error message for the other case as well. Similarly, if only one of the first two cases fail, both fail with the error message of the failed case.

I guess this happens even if there’s more than one test case with a multiline name.

Bug 2: Focusing

Focusing (prepending the test case’s name with f:) doesn’t work for test cases with multiline names. In essence, every test case runs even if I focus a test case with a multiline name.

I guess the same holds true for bangs (prepending the test case’s name with !).

[sksamuel]

Fix will be included in 4.0.6 and 4.1.0

[sksamuel]

This is a gradle limitation whereby it’s taking only the first line from the multi line, which is of course just a ‘\n’ in this case, so they clash. We should strip out new lines from multiline strings, as they won’t work in intellij or gradle. We already strip newlines in the intellij plugin.

On Sat, 16 May 2020 at 20:38, Neel Kamath [e-mail] wrote:

@ashishkujoy https://github.com/ashishkujoy Regarding Bug 1

Note: When I refer to “test case”, I mean “test case with a multiline name”.

I tried to reproduce bug 1 today with simpler code.

When one test case is supposed to pass, and another is supposed to fail, only one fails. This is not a bug.

However, bug 1 still exists, in the other scenario I mentioned. Consider the following code.

import io.kotest.core.spec.style.StringSpecimport io.kotest.matchers.shouldBe class MyTest : StringSpec({ “”" test case 1 “”" { 1 shouldBe 2 }

"""
test
case
2
""" { true shouldBe false }

})

Both tests are supposed to fail, and they both do. However, the test report shows test case 1’s failure for test case 2’s failure as well. Please unzip the attached folder and open index.html in your browser to see the Gradle report (I’m unsure whether this is a Gradle bug).

Test Report.zip https://github.com/kotest/kotest/files/4639514/Test.Report.zip Regarding Bug 2

Your focusing technique works, and makes sense. It didn’t occur to me that whitespace would’ve affected it, especially since Gradle replaces multiline test case names with single spaces. It might help to add this to the focus (and bang) docs.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/kotest/kotest/issues/1441#issuecomment-629729701, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFVSGXALS47EZY5FFIJNATRR4525ANCNFSM4NC3HDZA .