What is the best way to change the dims for a dense_vector in an index?

I have an index with a dense_vector field with 256 dimensions. I want to change to a new embedding model that uses a different number of dimensions. I dont mind losing the data in the old field. What's the best way to keep the data (other than the dense_vector) and change the dims?

I've tried:

  1. Looking into deleting a field -- but this appears to be impossible in Elastic.
  2. Creating a new index with the new mapping with a higher dimensional dense_vector and then reindexing from old to new. This failed during reindex because the dimensions of that field didn't match up.
  3. Creating a new index that was missing the dense_vector field, and then reindexing from old to new. This worked for reindexing, but then when I wanted to add the field to the mapping, it failed with mapper [source_title_vector] cannot be changed from type [float] to [dense_vector]...probably because the field exists in the data without a mapping (I guess its treated as a float?)

Do I need to write some code to go through the documents one at a time and insert them without this field? Any help would be appreciated.

Welcome to our community! :smiley:

You cannot change a mapping type once it has been created in the index. To do that you will need to create a new index with the mapping you want, then reindex into it, which is option 2 above.

I don't know much about dense vectors, but perhaps if you can share your reindex request, mappings and the error we can assist further

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.