# Finding unique values of a field from the returned search results

**URL:** <https://discuss.elastic.co/t/finding-unique-values-of-a-field-from-the-returned-search-results/32752>\
**Category:** Elasticsearch\
**Created:** [October 22, 2015, 8:33am UTC](https://discuss.elastic.co/t/finding-unique-values-of-a-field-from-the-returned-search-results/32752 "2015-10-22T08:33:32Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![apanimesh061](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/apanimesh061/32/927_2.png) [@apanimesh061](https://discuss.elastic.co/u/apanimesh061)\
**Post date:** [October 22, 2015, 8:33am UTC](https://discuss.elastic.co/t/finding-unique-values-of-a-field-from-the-returned-search-results/32752/1 "2015-10-22T08:33:32Z")

</div>

When I send a search request, I get a document of this format:

```
        {
            "_index": "texts_cluster",
            "_type": "documents",
            "_id": "5732",
            "_score": 1.8980495,
            "_source": {
               "bucket_id": "12345",
               "date": "2009-06-03",
               "text": "some large text",
            },
         }

```

I send a request so that ~1000 documents are returned. I want the unique `bucket_id`s of all the returned documents and then send search query according to the collected `bucket_id`s like:

```
POST /texts_cluster/_search
{
  "query": {
    "match": {
      "bucket_id": "12345"
    }
  },
  "sort": [
    {
      "date": {
        "order": "asc"
      }
    }
  ]
}

```

I thought of using the `aggregations` but they don't return the result in particular relevance order and also I only need the `bucket_id`s that occur in the 1000 documents returned by the search query.

One way that I can think of is to iterate through the 1000 documents and add the `bucket_id`s to a list as I wish to maintain the sequence. The append to the list would be only for a unique element. Is there any other easier method??

Thanks!

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [October 22, 2015, 9:23am UTC](https://discuss.elastic.co/t/finding-unique-values-of-a-field-from-the-returned-search-results/32752/2 "2015-10-22T09:23:26Z")

</div>

> [@apanimesh061](#):
>
> I thought of using the aggregations but they don't return the result in particular relevance order and also I only need the bucket\_ids that occur in the 1000 documents returned by the search quer

In the 2.0 release (RC1 is available now) we introduce the `sampler` aggregation that allows you to focus aggs on only the top-matching docs. Example below:

```
DELETE test
POST test/doc
{
	"bucket":1,
	"text":"we are one"
}
POST test/doc
{
	"bucket":1,
	"text":"one love"
}
POST test/doc
{
	"bucket":1,
	"text":"one time"
}
POST test/doc
{
	"bucket":2,
	"text":"two is company"
}
POST test/doc
{
	"bucket":2,
	"text":"takes two to tango"
}
GET test/doc/_search
{
   "query": {
	  "match": {
		 "text": "one"
	  }
   },
   "size": 0,
   "aggs": {
	  "bestDocs": {
		 "sampler": {
			"field": "bucket",
			"shard_size": 1000
		 },
		 "aggs": {
			"bestBuckets": {
			   "terms": {
				  "field": "bucket",
				  "size": 10
			   }
			}
		 }
	  }
   }
}

```

---

<div class="post-metadata">

**Author:** ![apanimesh061](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/apanimesh061/32/927_2.png) [@apanimesh061](https://discuss.elastic.co/u/apanimesh061)\
**Post date:** [October 22, 2015, 9:31am UTC](https://discuss.elastic.co/t/finding-unique-values-of-a-field-from-the-returned-search-results/32752/3 "2015-10-22T09:31:33Z")

</div>

Thanks for that but I am surrently using ES 1.7.3. Is there a solution for that in this version?

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [October 22, 2015, 9:45am UTC](https://discuss.elastic.co/t/finding-unique-values-of-a-field-from-the-returned-search-results/32752/4 "2015-10-22T09:45:19Z")

</div>

In \<2.0 you can either look at hits (the top-ranked docs) or aggs (summaries performed on _all_ docs) so if you want a summary of only the best docs your client code would have to trawl through hits (expensive) or you may be able to resort to some hacky groovy script [1] that tried to use PriorityQueues etc to limit stats to the top-matching docs.

In 2.0 the sampler agg makes this trivial.

Cheers  
Mark

[1] [https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-scripted-metric-aggregation.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-scripted-metric-aggregation.html)

---

<div class="post-metadata">

**Author:** ![apanimesh061](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/apanimesh061/32/927_2.png) [@apanimesh061](https://discuss.elastic.co/u/apanimesh061)\
**Post date:** [October 29, 2015, 4:19am UTC](https://discuss.elastic.co/t/finding-unique-values-of-a-field-from-the-returned-search-results/32752/5 "2015-10-29T04:19:55Z")

</div>

I upgraded to ES 2.0.0.

The query always returns the bucket ids in ascending order rather than in the order they appeared in the search results!!

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [October 29, 2015, 9:25pm UTC](https://discuss.elastic.co/t/finding-unique-values-of-a-field-from-the-returned-search-results/32752/6 "2015-10-29T21:25:29Z")

</div>

Try this then:

```
GET test/doc/_search
{
   "query": {
	  "match": {
		 "text": "one"
	  }
   },
   "size": 0,
   "aggs": {
	  "bestDocs": {
		 "sampler": {
			"field": "bucket",
			"shard_size": 1000
		 },
		 "aggs": {
			"bestBuckets": {
			   "terms": {
				  "field": "bucket",
				  "size": 10,
				  "order":{
					  "bestScore":"desc"
				  }
			  
			   },
			   "aggs":{
				   "bestScore":{
					   "max":{
						   "script":"_score"
					   }
				   }
			   }
			}
		 }
	  }
   }
}
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 11:41pm UTC](https://discuss.elastic.co/t/finding-unique-values-of-a-field-from-the-returned-search-results/32752/7 "2017-07-05T23:41:51Z")

</div>