Improve vector quantization API

Description

Follow-up of https://github.com/apache/lucene/pull/11860#discussion_r1027106953: the API for quantization of vectors is a bit surprising at times in that you can index bytes but then still get float[] arrays whose values are bytes at search time. Can we make sure it’s either bytes all the way, or floats all the way?

I’m not entirely sure how best to move it forward. One option would be to have a disjoint set of APIs for the binary encoding, like for doc values: different Field class, different VectorValues class, different searchNearestNeighbors method. I wonder if another option would be to make it floats all the way all the time, including for the byte encoding, ie. the Field and VectorValues class would still take and expose floats but VectorValues#binaryValue would be removed and the Field constructor would barf if the float values do not represent exact bytes.

[rmuir]

I would tend to expect some organization like this:

indexing:

• ByteVectorField
• FloatVectorField

searching:

• ByteVectorField.newQuery
• FloatVectorField.newQuery

codec api:

• KnnVectorsReader.getByteVectors(String field)
• KnnVectorsReader.getFloatVectors(String field)

[benwtrent]

OK, step one of this is done with the KnnByteVectorQuery. I am next approaching a new KnnByteVectorField and this will cause some refactoring to LeafReader or VectorValues.

Right now, VectorValues assumes all vector values are float[] and “expands” the bytes, needlessly.

There are many ways to approach this and I am honestly not sure the way to go here (being unfamiliar with typical API patterns).

It seems there are a handful of seemingly valid options:

• 1. Add a method to VectorValues (which is returned via LeafReader) that explicitly returns byteVectorValue() and will throw if vectorValue() is called when the encoding is BYTE (and throw on byteVectorValue() if encoding is FLOAT32).
  • This doesn’t seem like the best to me. Though we already require users to choose the correct methods based on the vector encoding.
• 1. Add a new AbstractVectorValues<T> where vectorValue() returns T. LeafReader would be changed to return that instead of VectorValues.
  • There don’t seem to be precedent on using generics like this anywhere, and this breaks with how we handle DocValues. The main difference with vectors and doc values is that numeric doc values are all returned as Long, but really could be any number whose bytes FIT in a long…
• 1. Add a new LeafReader#getByteVectorValues() that returns a ByteVectorValues object. The main downside here is that we are not making it flexible for adding new vector kinds in the future. We will want to add boolean vectors & hamming distance in the future. These are important for image search.
• 1. Simply make VectorValues always return BytesRef for everything and provide similarity that knows the encoding and can decode the float[] if that is the encoding. This feels closer to what we do with doc values (taking DoubleDocValuesField & DoubleValuesSource as prior art with how that interaction is done with NumericDocValues).

@jpountz @rmuir do y’all have opinions here?