ArgumentNullException with BlendedInfixSuggester (v. 4.8.0-beta00014)

Good day.

We ran into an error when trying DoLookup with some specific key values.

We have a dictionary with single item that has key “D|B©”. Then we try DoLookup for value “B©”. It results in an error:

System.ArgumentNullException
  HResult=0x80004003
  Message=Value cannot be null. (Parameter 'value')
  Source=System.Private.CoreLib
  StackTrace:
   at System.String.StartsWith(String value, StringComparison comparisonType)
   at Lucene.Net.Search.Suggest.Analyzing.BlendedInfixSuggester.CreateCoefficient(IndexSearcher searcher, Int32 doc, ICollection`1 matchedTokens, String prefixToken)
   at Lucene.Net.Search.Suggest.Analyzing.BlendedInfixSuggester.CreateResults(IndexSearcher searcher, TopFieldDocs hits, Int32 num, String key, Boolean doHighlight, ICollection`1 matchedTokens, String prefixToken)
   at Lucene.Net.Search.Suggest.Analyzing.AnalyzingInfixSuggester.DoLookup(String key, IEnumerable`1 contexts, Int32 num, Boolean allTermsRequired, Boolean doHighlight)
   at Lucene.Net.Search.Suggest.Analyzing.BlendedInfixSuggester.DoLookup(String key, IEnumerable`1 contexts, Int32 num, Boolean allTermsRequired, Boolean doHighlight)
   at Lucene.Net.Search.Suggest.Analyzing.AnalyzingInfixSuggester.DoLookup(String key, IEnumerable`1 contexts, Boolean onlyMorePopular, Int32 num)
   at Lucene.Net.Search.Suggest.Analyzing.BlendedInfixSuggester.DoLookup(String key, IEnumerable`1 contexts, Boolean onlyMorePopular, Int32 num)
   at Lucene.Net.Search.Suggest.Lookup.DoLookup(String key, Boolean onlyMorePopular, Int32 num)
   at LucenNetIssue.Program.Main(String[] args) in C:\Users\monco\source\repos\LucenNetIssue\Program.cs:line 39

Code to reproduce.

using System;
using System.Collections.Generic;

using Lucene.Net.Analysis.Standard;
using Lucene.Net.Search.Suggest;
using Lucene.Net.Search.Suggest.Analyzing;
using Lucene.Net.Store;
using Lucene.Net.Util;

namespace LucenNetIssue
{
    class Program
    {
        static void Main(string[] args)
        {
            const LuceneVersion LUCENE_VERSION = LuceneVersion.LUCENE_48;

            var analyzer = new StandardAnalyzer(LUCENE_VERSION);

            var suggester = new BlendedInfixSuggester(LUCENE_VERSION,
                new MMapDirectory("search"),
                analyzer,
                analyzer,
                4,
                BlendedInfixSuggester.BlenderType.POSITION_LINEAR,
                1
            );

            var items = new[]
            {
                new PlaceItem
                {
                    Key = "D|B(C)"
                }
            };

            suggester.Build(new PlaceIterator(items));

            var result = suggester.DoLookup("B(C)", false, 5);
        }
    }

    class PlaceItem
    {
        public string Key { get; set; }
    }

    internal class PlaceIterator : IInputEnumerator
    {
        private readonly IEnumerator<PlaceItem> _enumerator;

        public BytesRef Current => _enumerator.Current is { } ? new BytesRef(_enumerator.Current.Key) : null;

        public IComparer<BytesRef> Comparer => null;
        public long Weight => 0;
        public BytesRef Payload => null;
        public bool HasPayloads => true;
        public ICollection<BytesRef> Contexts => Array.Empty<BytesRef>();
        public bool HasContexts => true;

        public PlaceIterator(IEnumerable<PlaceItem> placeItems)
        {
            _enumerator = placeItems.GetEnumerator();
        }

        public bool MoveNext() => _enumerator.MoveNext();
    }
}

TargetFramework: net5.0 Packages: Lucene.Net Version=“4.8.0-beta00014”, Lucene.Net.Suggest Version=“4.8.0-beta00014”.

Thanks for your attention.

UPD. If we change Key “D|B©” to “A|B©” or “B|B©” or "C|B©"then DoLookup will work.

[NightOwl888]

@rclabo - Thanks for helping me debug this in Java.

[NightOwl888]

@Maxwellwr

Thanks for the info. I had a look at the LUCENE-6093 patch and it looks like a pretty straightforward fix. We aren’t going out of our way to bring in changes from newer versions of Lucene, but if they affect usability, we are accepting PRs for patches from the community.

Here is the source to use if you need more context than the patch file: Lucene 4.10.4 suggest source code