Improve wn.ic documentation and error messages regarding unsmoothed frequencies

While trying to create a list of similarities between synsets I’m running lin similarity on each element of a list, so basically:

first_word = wn.synsets(token.text, pos=wordnet_pos)[0]
hyponyms_ic = [(hyponym, lin(first_word, hyponym, wn_ic)) for hyponym in first_word.hyponyms()]

This throws a ValueError in the ic.py file line 32 (as of right now):

return -log(synset_probability(synset, freq))

[goodmami]

Is there any disadvantages for using an IC file with the -add1 smoothing?

I don’t think so. It prevents these kind of divide-by-zero and other math domain errors, and I don’t think anyone cares about the actual frequencies of the words. The -resnik version is probably good, too, as it gives weight proportional to the number of senses for a word. I’m not sure which performs empirically better, if any does.

If not, then perhaps the documentation could default to […]

Yes, that’s probably a good idea.

Either that or maybe an Exception for the ic.py code which explains why the ValueError was thrown and what can be done about it.

Yeah, maybe best to do both. Not everyone reads the documentation 😃

Thanks for the feedback!

[goodmami]

@M49ICKPIxi3 it took me a while to get to this but I’ve now updated the wn.ic documentation so the example shows the file with smoothing and there’s a short explanation in the text.

I did not introduce a custom exception on the ValueError because I could no longer recreate the error with the latest version of the software and data, and therefore I couldn’t test it out. If some example persists with the newest software and data, I’d probably add in the custom error message. But for now I think the documentation fix is sufficient.