Infinite loop in XOR filter construction

The following invocation leads to an infinite loop:

    XorPlus8.construct(new long[]{0xef9bddc5166c081cL, 0x33bf87adaa46dcfcL});

It appears that with these inputs, getHash(key, seed, index) always returns index. As a result, the two values always collide on all three of their table slots (regardless of the seed), and hence there are no single-occupied table slots and the construction algorithm never terminates. I am scratching my head at this; it seems impossible that the hash algorithm could behave in such a fashion regardless of seed. But this is the behavior I’ve observed. Any thoughts on why this might be?

Regardless of exactly what is happening here, for safety purposes, it would seem like a good idea to implement a maximum retry count for table construction (after which an exception would be thrown).

I will try to dig into this further. If I’m able to determine what is happening and implement a fix, would you be open to a pull request?

[steve-scalyr]

As a better-than-nothing fallback, what would you think of throwing an exception after some number of attempts to populate the table? Then the client could at least take some action to compensate (e.g. don’t create an XOR filter for this particular data set, and instead assume that all possible values might be present). It would also be an opportunity to provide a diagnostic message.

I can put together a pull request, if you like.

[steve-scalyr]

Done in https://github.com/FastFilter/fastfilter_java/pull/30.