Dynamic mapping of dense_vector

Elastic Stack Elasticsearch
1

I have a float which I want to map to dense_vector type field. I want to use dynamic mapping but it is dynamically always mapped to float type. Is there a way to map to dense_vector type when I am using dynamic mapping for creating the index ?

2

According to this thread dynamic mapping of vectors was added in Elasticsearch 8.11. Are you by any chance on an earlier version?

3

As @Christian_Dahlqvist said, dynamic mapping of dense vectors was introduced in 8.11.0 with float arrays > size 128 dynamically mapped to dense_vector. If you're using a version >8.11.0 and seeing an issue with this dynamic mapping, it would be helpful to see an example of what you're sending in that's not being mapped correctly. Thank you!

4

I am using ES version 8.13.2, inserting dense_vector (double) data using java client. It creates index with dynamic mapping but type of my_vector field is "float". I want my_vector field type to be "dense_vector" with dynamic mapping.

import org.apache.http.HttpHost;
import org.elasticsearch.client.RestClient;

import co.elastic.clients.elasticsearch.ElasticsearchClient;
import co.elastic.clients.elasticsearch._types.ElasticsearchException;
import co.elastic.clients.json.jackson.JacksonJsonpMapper;
import co.elastic.clients.transport.ElasticsearchTransport;
import co.elastic.clients.transport.endpoints.BooleanResponse;
import co.elastic.clients.transport.rest_client.RestClientTransport;
import co.elastic.clients.elasticsearch.core.*;

public class ElasticsearchUtil {
	
	private ElasticsearchClient client;
	private BulkRequest.Builder builder = new BulkRequest.Builder();
	
	public void connect() {
		RestClient restClient = RestClient.builder(new HttpHost("localhost", 9200)).build();
		ElasticsearchTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper());
		client = new ElasticsearchClient(transport);
	}

	public void insert(DenseVec vec) {
		try {
			client.index(i -> i
					.index("my-index")
					.id("1")
					.document(vec));
			System.out.println("Successfully inserted");
			
		} catch (ElasticsearchException | IOException e) {
			e.printStackTrace();
		}
	}

	public static void main(String[] args) {
		ElasticsearchUtil util = new ElasticsearchUtil();
		util.connect();
		util.insert(new DenseVec("test data", new double[] {1234567.5678878, 1234567.2378665, 1234567.85987986}));
	}
}

class DenseVec {
	private double[] my_vector;
	private String my_text;
	
	public DenseVec(String my_text, double[] my_vector) {
		this.my_vector = my_vector;
		this.my_text = my_text;
	}
	
	public double[] getMy_vector() {
		return my_vector;
	}
	public void setMy_vector(double[] my_vector) {
		this.my_vector = my_vector;
	}
	public String getMy_text() {
		return my_text;
	}
	public void setMy_text(String my_text) {
		this.my_text = my_text;
	}
}

The mapping of dynamically created index is :-

curl -X GET "localhost:9200/my-index/_mappings?pretty"
{
  "my-index" : {
    "mappings" : {
      "properties" : {
        "my_text" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "my_vector" : {
          "type" : "float"  -----> (I want this to be dense_vector type)
        }
      }
    }
  }
}
5

What is the length of your vector?

6

In above example it is of length 3....but it may contain variable length vectors.

7

As Kathleen pointed out, the dynamic mapping only maps arrays with a length greater than 128 as vectors. If you have shorter vectors you need to map them explicitly.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.