Poor numeric domain coverage when using default arbitraries

Testing Problem

Poor numeric domain coverage when using default arbitraries. The property evenNumbersAreEvenAndSmall is not falsified by Jqwik 0.8.4 using the default jqwik.poperties in the tests below:

public class JqwikTests {

    public static final Condition<Long> EVEN = new Condition<>(x -> x % 2 == 0, "even");
    public static final Condition<Long> SMALL = new Condition<>(x -> x < 2000, "< 2000");

    @Provide
    Arbitrary<Long> evenNumbers() {
        return Arbitraries.longs().filter(l -> l % 2 == 0);
    }

    @Property
    void evenNumbersAreEven(@ForAll("evenNumbers") long evenNumber) {
        assertThat(evenNumber)
                .is(EVEN);
    }

    @Property
    void evenNumbersAreEvenAndSmall(@ForAll("evenNumbers") long evenNumber) {
        System.out.println(evenNumber);
        assertThat(evenNumber)
                .is(EVEN)
                .is(SMALL);
    }
}

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running JqwikTests
0
-9223372036854775808
8
-468
-214
284
186
... many more (993? :-)) numbers, but all roughly in the region (-500, 500)

The other tested frameworks (JunitQuickCheck, QuickTheories, VavrTest) are able to falsify this using default settings but have unusable or no shrinking.

Manually setting the range to (Long.MIN_VALUE + 1, Long.MAX_VALUE - 1) solves this as does increasing the number of tries.

Suggested Solution

The very small test domain (defaultMaxFromTries = Math.max(tries / 2 - 3, 3))) should either be documented quite prominently or increased (at least for int/long).

Discussion

Documenting does not need any further discussion I guess 😃

Increasing the test domain while keeping the number of tries obviously spreads out the sample data which might be wanted–or not.

[jlink]

I chose a different path. Random number generation now uses the whole domain between allowed min and max values. However, the full domain is divided into several partitions so that lower numbers are generated with a higher probability. The cut off point is determined by Math.max(tries / 2, 10).

@rkraneis I tried with your original example and evenNumbersAreEvenAndSmall is now successfully falsified.

[jlink]

Change available in version 0.8.5-SNAPSHOT