Dictionary word not always recognized

In this example I have configured nbvcxz with an additional single-entry dictionary to test how a common password occurrence (surname or given name plus a few numbers) is handled:

// Just a random string for testing. Please assume for the sake of the example that
// 'Gohklhiepu' is the user's surname.
Map<String, Integer> excludeMap = new HashMap<>();
excludeMap.put("Gohklhiepu", 0);

// Just like the README.
List<Dictionary> dictionaryList = ConfigurationBuilder.getDefaultDictionaries();
dictionaryList.add(new Dictionary("exclude", excludeMap, true));

Configuration configuration = new ConfigurationBuilder()
        .setDictionaries(dictionaryList)
        .setMinimumEntropy(30d)
        .createConfiguration();

Nbvcxz nbvcxz = new Nbvcxz(configuration);


// Test A.

Result result = nbvcxz.estimate("Gohklhiepu");

System.out.println(result.getEntropy());
// 0.0

for (Match match : result.getMatches()) {
    System.out.println(match.getDetails());
    // DictionaryMatch
}


// Test B.

result = nbvcxz.estimate("Gohklhiepu3425");

System.out.println(result.getEntropy());
// 60.29210956096036

for (Match match : result.getMatches()) {
    System.out.println(match.getDetails());
    // A series of BruteForceMatch
}

As expected, using the fictional dictionary word as password gives 0.0 entropy.

Surprisingly, word + 3425 (which is a rather weak password if we assume that word is the user’s surname and the number his postal code or house number) results in a rather high entropy of 60.3. It looks like the word is not recognized as a dictionary word — all matches are BruteForceMatch.

Could this be a bug?

[Tostino]

Alright, I have committed a fix in: 0bcab0c68a8f234c737fbe1afedab434f4dd2f51

There was one extra problem I had to tackle, because it wasn’t obvious but was causing another issue after the first I found. And that problem is that dictionaries need to only contain lower case words for the matching to work properly. We do case insensitive matching by just making sure everything is being lower cased to speed things up. That was not at all obvious with the previous API. I created a builder class for creating dictionaries, which will help with that. The readme is updated with an example of using that builder.

If you want to use the constructor directly, I made sure to update the javadoc to note that words had to be lower cased to work correctly.

Also created some tests for catching issues with exclusion dictionaries in the future.

I pushed out 1.3.3 to Maven, so so that should be out there shortly. Will create another release asap.

Please let me know if you find anything else working not as you’d expect.

Thanks, -Adam

[jdhoek]

The second example from my test now gives a much lower 16.421653954955087 as entropy. Looks good!