# Getting aggregate statistics about a dense\_vector field **URL:** \ **Category:** Elasticsearch\ **Created:** [March 29, 2021, 7:30pm UTC](https://discuss.elastic.co/t/getting-aggregate-statistics-about-a-dense-vector-field/268694 "2021-03-29T19:30:10Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![MaJaHa95](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/majaha95/32/26121_2.png) [@MaJaHa95](https://discuss.elastic.co/u/MaJaHa95)\ **Post date:** [March 29, 2021, 7:30pm UTC](https://discuss.elastic.co/t/getting-aggregate-statistics-about-a-dense-vector-field/268694/1 "2021-03-29T19:30:11Z")
Hey, I'm making lots of great use of the `dense_vector` field type, but I've hit a bit of a wall now. Is there any way I can get the "average" from a vector field? ## Context My real use-case is a bit more complex, but imagine I have this mapping for emails with a spam flag: ```auto PUT /emails { "mappings": { "properties": { "text": { "type": "text" }, "text_vector": { "type": "dense_vector", "dims": 300 }, "is_spam": { "type": "boolean" } } } } ``` I've vectorized and ingested my data, so I now have 150,000 docs that each have 300-dimension vectors associated with them. Now, I want to do a vector similarity search (using `cosineSimilarity`) to find documents similar to those which I've flagged as spam, but which are not yet flagged. With a query vector in-hand, this is made easy by the [vector function scoring functionality](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-script-score-query.html#vector-functions), and I can quickly retrieve the most similar documents. ```auto POST /emails/_search { "query": { "script_score": { "query" : { "bool" : { "must_not" : { "term" : { "is_spam" : true } } } }, "script": { "source": "cosineSimilarity(params.query_vector, 'text_vector') + 1.0", "params": { "query_vector": [...] // this is lengthy } } } } } ``` But I need a way to build out that `query_vector` initially. This works well if I have seed text, or a seed document, because I can calculate the vector (or fetch it from the source) as a prerequisite, but when I want to cover a collection of documents, it gets more difficult. As an initial stab, I want to try and get the "center" of the flagged documents. To find this, I simply need the item-wise mean vector. So if I have these documents: ```auto 1: { ..., "text_vector": [1, 2, 3], ... }, 2: { ..., "text_vector": [2, 1, 3], ... }, 3: { ..., "text_vector": [0, 5, 1], ... } ``` Then I can just sum up each array index, then divide by three: ```auto sum[0] = 1 + 2 + 0 = 3 sum[1] = 2 + 1 + 5 = 8 sum[2] = 3 + 3 + 1 = 7 avg[0] = 3 / 3 = 1 avg[1] = 8 / 3 = 2.67 avg[2] = 7 / 3 = 2.33 ``` And so, I pass in `"query_vector": [1, 2.67, 2.33]` to the above query, and retrieve the documents I'm looking for. ## Current approach Right now, I start with a paginated or scroll query to get back the document sources, then build up the sums and averages procedurally. This is fine for a few documents, but it can be quite troublesome when looking at more and more. At the very least, there's a lot of network traffic required to read out 300 dimensions from several documents. ## Desired approach I'd love to be able to use the `avg` aggregation to get these values handed to me. Obviously, a single aggregation isn't going to give me back an array result, but I would much prefer something like: ```auto POST /emails/_search { "size": 0, "query": { "bool" : { "must_not" : { "term" : { "is_spam" : true } } } }, "aggs": { "text_vector_0": { "avg": { "field": "text_vector.0" } }, "text_vector_1": { "avg": { "field": "text_vector.1" } }, ... "text_vector_299": { "avg": { "field": "text_vector.299" } }, } } ``` But of course, as expected, those fields don't exist. ## Options ### Read the values in a script I had thought to use script aggregations, but any attempt to read elements from the `dense_vector` field return the error: > unsupported\_operation\_exception: accessing a vector field's value through 'get' or 'value' is not supported ### Use the dot product to retrieve values I came up with another (I think, clever) idea to use the `dotProduct` function with identity matrix slices, but I quickly found that [`dotProduct`, `cosineSimilarity`, and other vector functions are only available in the score context](https://www.elastic.co/guide/en/elasticsearch/painless/7.12/painless-api-reference-score.html#_static_methods_2). So now the best I've got is a multi-search that aggregates score for the query vectors individually: ```auto POST /_msearch=responses.aggregations.text_vector_*.value { "index": "emails" } { "size": 0, "query": { "script_score": { "query": { "match_all": {} }, "script": { "source": "dotProduct(params.query_vec, 'text_vector') + 1.0", "params": { "query_vec": [1, 0, 0] } } } }, "aggs": { "text_vector_0": { "avg": { "script": { "source": "return _score.floatValue() - 1;" } } } } } { "index": "emails" } { "size": 0, "query": { "script_score": { "query": { "match_all": {} }, "script": { "source": "dotProduct(params.query_vec, 'text_vector') + 1.0", "params": { "query_vec": [0, 1, 0] } } } }, "aggs": { "text_vector_1": { "avg": { "script": { "source": "return _score.floatValue() - 1;" } } } } } { "index": "emails" } { "size": 0, "query": { "script_score": { "query": { "match_all": {} }, "script": { "source": "dotProduct(params.query_vec, 'text_vector') + 1.0", "params": { "query_vec": [0, 0, 1] } } } }, "aggs": { "text_vector_2": { "avg": { "script": { "source": "return _score.floatValue() - 1;" } } } } } ... ``` But like, that sucks. Real bad. Remember, I've got 300-dimension vectors, which adds searches to this, and lengthens the query vectors themselves. ### Store the vectors separately This is the obvious choice, but has its own challenges. In my real use-case, I'm dealing with more than one dense vector. So I have to multiply work multiple times. I also can't use `copy_to` because of how it works with arrays, so I either have to start using ingestion pipelines, or do it as part of my procedural code. And even once I have a copy, the doc values `.get(0)` doesn't guarantee order, so that doesn't work. What I have to do, it seems, is store the vectors as objects, adding a dynamic template to the above mapping like: ```auto PUT /emails { "mappings": { "dynamic": "strict", "dynamic_templates": [ { "raw_vector": { "path_match": "text_vector_obj.*", "mapping": { "type": "float" } } } ], "properties": { "text": { "type": "text" }, "text_vector": { "type": "dense_vector", "dims": 300 }, "is_spam": { "type": "boolean" }, "text_vector_obj": { "type": "object", "dynamic": true } } } } ``` I'm then able to add rows as arrays and dictionaries. ```auto 3: { ..., "text_vector": [0, 5, 1], "text_vector_obj": { "0": 0, "1": 5, "3": 1 }, ... } ``` This works, and indeed makes use of the `avg` aggregation with a simple `field` argument quite trivial, but it's tremendously clunky for the ingestion, not to mention how it requires values to be stored in multiple fields. ## Alternatives? So, are there any alternatives out there? I recognize that I've covered a couple dead-ends here that could make feature requests over on GitHub, but I wanted to sanity check my options before going to dev. I wonder if there's a cleaner way to be performing this. ---
**Author:** ![MaJaHa95](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/majaha95/32/26121_2.png) [@MaJaHa95](https://discuss.elastic.co/u/MaJaHa95)\ **Post date:** [March 29, 2021, 9:53pm UTC](https://discuss.elastic.co/t/getting-aggregate-statistics-about-a-dense-vector-field/268694/2 "2021-03-29T21:53:34Z")
Worth noting, the cluster I'm on happens to be 7.8, but it's not a huge deal if I have to update it. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 26, 2021, 9:54pm UTC](https://discuss.elastic.co/t/getting-aggregate-statistics-about-a-dense-vector-field/268694/3 "2021-04-26T21:54:06Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed. ---
**Author:** ![Quentin\_Pradet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/quentin_pradet/32/94192_2.png) [@Quentin\_Pradet](https://discuss.elastic.co/u/Quentin_Pradet)\ **Post date:** [August 31, 2022, 7:56am UTC](https://discuss.elastic.co/t/getting-aggregate-statistics-about-a-dense-vector-field/268694/4 "2022-08-31T07:56:31Z")
---
**Author:** ![Quentin\_Pradet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/quentin_pradet/32/94192_2.png) [@Quentin\_Pradet](https://discuss.elastic.co/u/Quentin_Pradet)\ **Post date:** [August 31, 2022, 8:04am UTC](https://discuss.elastic.co/t/getting-aggregate-statistics-about-a-dense-vector-field/268694/5 "2022-08-31T08:04:51Z")
@MaJaHa95 The three options you mentioned are spot on. We don’t currently have a great solution to get the average of dense vectors. Note however that the first option is in fact possible with the `doc[].vectorValue` function ([Script score query | Elasticsearch Guide [8.4] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-script-score-query.html#vector-functions)). Here's an example `scripted_metric` aggregation to do the avg on dense\_vector type: ```json { "query": { "match_all": {} }, "aggs": { "profit": { "scripted_metric": { "init_script": "state.vs = []", "map_script": "state.vs.add(doc['my_vector'].vectorValue)", "combine_script": "int dims = state.vs[0].length; double[] sum_v = new double[dims]; for (v in state.vs) {int dim = 0; while (dim < dims) {sum_v[dim] += v[dim]; dim = dim + 1}} return sum_v", "reduce_script": "int dims = states[0].length; double[] sum_v = new double[dims]; for (v in states) {int dim = 0; while (dim < dims) {sum_v[dim] += v[dim]; dim = dim + 1}} return sum_v" } } } } ``` But keep in mind, this aggregation will be slow and takes a lot of heap memory. (Thanks @mayya for the help and the snippet!) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 4, 2022, 8:27am UTC](https://discuss.elastic.co/t/getting-aggregate-statistics-about-a-dense-vector-field/268694/6 "2022-11-04T08:27:41Z")