Describing assertions

Let me preface this by saying that I’m relatively inexperienced with unit testing, Kotlin and Kluent, so this is more of a discussion question than a specific bug or feature recommendation.

In starting to experiment with Spek and Kluent to write some tests for my Java projects, I’ve struggled with wanting to provide more context than perhaps is necessary. I get an assertion stack trace and I see the values and can use the trace to find the line that failed, but I don’t have much about the specific test. That’s because I haven’t added highly granular describe/on/it descriptions, but I also feel like it’s probably overkill or redundant to add those for every assertion. For example, I have a financial object with several amount properties pertaining to a given situation. I’ve structured the test to describe the situation, but in my mind, the scope of the test is a group of assertions about the state of the object, not a bunch of tiny tests per assertion. But it would be nice to know at a glance which property failed.

describe("the scenario") {
	it("the test") {
		describe("additional description about this assertion") { // bad: won't work; eats the failure
			"foo" shouldEqual "bar"
		}
	}
}

I found that bound callable references to properties could be used to decorate bean property testing in a really satisfying way…

infix fun <T> KProperty0<T>.shouldEqual(value: T) = Assert.assertEquals(this.name, value, this.get())
...
with(MyBean()) {
	::foo shouldEqual "expected-value"
}

But this overloading conflicts with Kluent’s so it would have to use a different name or operator, and Kotlin unfortunately doesn’t yet support references to synthetic extension properties for Java set/get methods, so it’s of limited use for testing my Java code.

I followed a Jasmine issue some time ago where a similar assertion level motivation was expressed. It’s arguable (and was) that this should be expressed in the BDD framework or in better custom assertions, but I think there are cases where it’s natural to want to provide some context at the assertion level. Though it might not be practical to support value should... expected because "reason" in Kotlin due to the evaluation order. JUnit does seem to support descriptions in its assertions though.

I do see that Kluent has a should method, but it seems to put the whole burden of message construction on the caller rather than combining with or augmenting the normal assertions.

So, my impulse is to look for a concise way to decorate assertions. Is the answer in the context of Kluent:

• That this is wrong…the stacktrace and values are sufficient?
• That this is the BDD framework’s job; write better/more-granular specs?
• That JUnit-style assertion descriptions just aren’t a good fit for Kluent’s simple infix style?
• Something else I’m missing?

(Kluent is still very clever and has been a great demonstration of several Kotlin features, so thanks for that.)

[MarkusAmshove]

I think this is out of scope for Kluent, because we just want to provide some assertions, not being a spec framework 😃

Have a look at Spek or KotlinTest for different tastes of specification frameworks

[nlwillia]

Hey, thanks for the reply.

Spek isn’t super precise about its grammar, but I gather that they consider it the lowest level context.

The stack trace and comparison is definitely sufficient to isolate what’s wrong; it’s just nice as you compared with FluentAssertions to get more info.

I look forward to seeing what you come up with for properties. There’s the caveat I mentioned for synthetic Java properties (more of me exploring that here with limited success), but overall, property references seem like a useful addition.

For ad hoc description, I was thinking about a scoped hook for message decoration, but I realized that I can get there by intercepting the exception.

fun decorate(message: (String) -> String, block: () -> Unit) {
	try {
		block.invoke()
	} catch (e: AssertionError) {
		val e1 = AssertionError(message.invoke(e.message ?: ""))
		e1.stackTrace = e.stackTrace
		throw e1
	}
}

operator fun String.invoke(block: () -> Unit) = decorate({"$this: $it"}, block)
...
"custom prefix" {
	// assertions
}

fun because(message: String, block:() -> Unit) = decorate({"$it because $message"}, block)
...
because("I said so") {
	// assertions
}

Perhaps not an ideal practice, but it gets the job done if it’s detail that I don’t want to express at the spec level.