Change in dense_vector field indexation default in ES 8.14 - Existing indices behaviour?

Hello,

In the ES 8.14 patch notes, we can see that now, the dense_vector field type used for vector search will now be indexed by default with the quantized hnsw_int8 graphs.

Prior to that, hnsw_int8 was the graph used to index the field, only optionally. The default was plain hnsw.

On the upgrade of our cluster (from version 8.10, to version 8.14), we didn't observe any downtime nor any behavior change. We expect the existing indices to have their dense_vector fields still indexed as hnsw as it was the default at the time of the index creation & field mapping.

Will the future documents added to the index created in 8.10 have their dense_vector fields indexed as hnsw, like the previous documents ?

Would adding to the mapping a new dense_vector field to this old index now default to hnsw_int8 ?

Do I need to create a new index to observe the new behavior of hnsw_int8 being the default and hnsw being used only optionally ?

I couldn't find anything in the documentation covering how upgrading the clusters would impact or not the existing indices. But if there is some that would answer my confusions, please redirect me to it.

Regards,

Hi @clemsau !

The defaults for dense_vector fields depend on the Elasticsearch version that the index was created on.

That means that an index created prior to 8.14 will maintain hnsw as the default. An index that was created on 8.14 or afterwards, will use hnsw_int8 as the default.

So, adding a new dense_vector field to a pre-8.14 index should have the same defaults as before.

Hopefully that helps!

1 Like