Terms Aggregation and scope

Elastic Stack Elasticsearch
1

Dear list,

I need to create an aggregation by a specific field, named
"datum_uitspraak_ymd'. I am using the below curl command and it works fine
in a sense that it returns the aggregation listed below. While this result
seems OK enough, it seems that the keys listed in the aggregation are
limited to those listed in 10 records.

As the number of unique values for this key is much higher than 10, it
seems that the aggregation's scope is global as far as it searches for
documents with a value identical to one of the 10 listed, but it is limited
as far as the key values used in the aggregation is concerned.

How do I need to set up my curl command in order for the aggregation to
return more key-value pairs?

Many thanks,
Diederik

Command:
curl -XGET 'http://localhost:9200/_search?pretty=true' -d '{ "size": 0,
"aggs": { "datum_uitspraak_ymd": { "terms": { "field":
"datum_uitspraak_ymd" } } } }'

Returns:
"aggregations" : { "datum_uitspraak_ymd" : { "buckets" : [ { "key" :
"20121219", "doc_count" : 612 }, { "key" : "20120516", "doc_count" : 526 },
{ "key" : "20110601", "doc_count" : 472 }, { "key" : "20121218",
"doc_count" : 468 }, { "key" : "20090520", "doc_count" : 349 }, { "key" :
"20101222", "doc_count" : 274 }, { "key" : "20120711", "doc_count" : 272 },
{ "key" : "20090429", "doc_count" : 246 }, { "key" : "20120718",
"doc_count" : 230 }, { "key" : "20120425", "doc_count" : 226 } ] } }

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/34b8c9ca-8fe1-45ca-96b1-49b2a4f55696%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Diederik,

To increase the number of terms returned by the terms aggregation you will
need to add the 'size' parameter to your aggregation. The below curl
command will return you the top 200 terms (ordered by decending doc_count).

curl -XGET 'http://localhost:9200/_search?pretty=true' -d '{ "size": 0,
"aggs": { "datum_uitspraak_ymd": { "terms": { "field":
"datum_uitspraak_ymd", "size" : 200 } } } }'

You may also find the following link to the documentation useful regarding
the size parameter of the terms aggregation.

Hope this helps,

Colin

On Monday, 7 July 2014 13:09:52 UTC+1, Diederik Meijer wrote:

Dear list,

I need to create an aggregation by a specific field, named
"datum_uitspraak_ymd'. I am using the below curl command and it works fine
in a sense that it returns the aggregation listed below. While this result
seems OK enough, it seems that the keys listed in the aggregation are
limited to those listed in 10 records.

As the number of unique values for this key is much higher than 10, it
seems that the aggregation's scope is global as far as it searches for
documents with a value identical to one of the 10 listed, but it is limited
as far as the key values used in the aggregation is concerned.

How do I need to set up my curl command in order for the aggregation to
return more key-value pairs?

Many thanks,
Diederik

Command:
curl -XGET 'http://localhost:9200/_search?pretty=true' -d '{ "size": 0,
"aggs": { "datum_uitspraak_ymd": { "terms": { "field":
"datum_uitspraak_ymd" } } } }'

Returns:
"aggregations" : { "datum_uitspraak_ymd" : { "buckets" : [ { "key" :
"20121219", "doc_count" : 612 }, { "key" : "20120516", "doc_count" : 526 },
{ "key" : "20110601", "doc_count" : 472 }, { "key" : "20121218",
"doc_count" : 468 }, { "key" : "20090520", "doc_count" : 349 }, { "key" :
"20101222", "doc_count" : 274 }, { "key" : "20120711", "doc_count" : 272 },
{ "key" : "20090429", "doc_count" : 246 }, { "key" : "20120718",
"doc_count" : 230 }, { "key" : "20120425", "doc_count" : 226 } ] } }

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9f0cb2af-fa2d-40b0-9930-b884141f2969%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

3

Hi Colin,

Thank you, that works perfectly!

Is there any way to limit the key-value pairs by a certain parameter, in the example below: to limit the aggregation to "datum_uitspraak_ymd" keys that start with "2014"?

Or does that require the combination of a filter and an aggregation?

Many thanks,

Diederik

Op Jul 7, 2014, om 2:47 PM heeft Colin Goodheart-Smithe colin.goodheart-smithe@elasticsearch.com het volgende geschreven:

Diederik,

To increase the number of terms returned by the terms aggregation you will need to add the 'size' parameter to your aggregation. The below curl command will return you the top 200 terms (ordered by decending doc_count).

curl -XGET 'http://localhost:9200/_search?pretty=true' -d '{ "size": 0, "aggs": { "datum_uitspraak_ymd": { "terms": { "field": "datum_uitspraak_ymd", "size" : 200 } } } }'

You may also find the following link to the documentation useful regarding the size parameter of the terms aggregation.

Elasticsearch Platform — Find real-time answers at scale | Elastic

Hope this helps,

Colin

On Monday, 7 July 2014 13:09:52 UTC+1, Diederik Meijer wrote:
Dear list,

I need to create an aggregation by a specific field, named "datum_uitspraak_ymd'. I am using the below curl command and it works fine in a sense that it returns the aggregation listed below. While this result seems OK enough, it seems that the keys listed in the aggregation are limited to those listed in 10 records.

As the number of unique values for this key is much higher than 10, it seems that the aggregation's scope is global as far as it searches for documents with a value identical to one of the 10 listed, but it is limited as far as the key values used in the aggregation is concerned.

How do I need to set up my curl command in order for the aggregation to return more key-value pairs?

Many thanks,
Diederik

Command:
curl -XGET 'http://localhost:9200/_search?pretty=true' -d '{ "size": 0, "aggs": { "datum_uitspraak_ymd": { "terms": { "field": "datum_uitspraak_ymd" } } } }'

Returns:
"aggregations" : { "datum_uitspraak_ymd" : { "buckets" : [ { "key" : "20121219", "doc_count" : 612 }, { "key" : "20120516", "doc_count" : 526 }, { "key" : "20110601", "doc_count" : 472 }, { "key" : "20121218", "doc_count" : 468 }, { "key" : "20090520", "doc_count" : 349 }, { "key" : "20101222", "doc_count" : 274 }, { "key" : "20120711", "doc_count" : 272 }, { "key" : "20090429", "doc_count" : 246 }, { "key" : "20120718", "doc_count" : 230 }, { "key" : "20120425", "doc_count" : 226 } ] } }

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/wCkb_xHaUmQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9f0cb2af-fa2d-40b0-9930-b884141f2969%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/2CC2403E-C358-4405-AA70-3ECDE29BF9D2%40tenhorses.com.
For more options, visit https://groups.google.com/d/optout.

4

Glad it worked.

Yes, there are options for includes and excludes patterns. Take a look at
the following link for information on how to use them.

Colin

On Monday, 7 July 2014 14:04:56 UTC+1, Diederik Meijer wrote:

Hi Colin,

Thank you, that works perfectly!

Is there any way to limit the key-value pairs by a certain parameter, in
the example below: to limit the aggregation to "datum_uitspraak_ymd" keys
that start with "2014"?

Or does that require the combination of a filter and an aggregation?

Many thanks,

Diederik

Op Jul 7, 2014, om 2:47 PM heeft Colin Goodheart-Smithe <
colin.goodheart-smithe@elasticsearch.com> het volgende geschreven:

Diederik,

To increase the number of terms returned by the terms aggregation you will
need to add the 'size' parameter to your aggregation. The below curl
command will return you the top 200 terms (ordered by decending doc_count).

curl -XGET 'http://localhost:9200/_search?pretty=true' -d '{ "size": 0,
"aggs": { "datum_uitspraak_ymd": { "terms": { "field":
"datum_uitspraak_ymd", "size" : 200 } } } }'

You may also find the following link to the documentation useful regarding
the size parameter of the terms aggregation.

Elasticsearch Platform — Find real-time answers at scale | Elastic

Hope this helps,

Colin

On Monday, 7 July 2014 13:09:52 UTC+1, Diederik Meijer wrote:

Dear list,

I need to create an aggregation by a specific field, named
"datum_uitspraak_ymd'. I am using the below curl command and it works fine
in a sense that it returns the aggregation listed below. While this result
seems OK enough, it seems that the keys listed in the aggregation are
limited to those listed in 10 records.

As the number of unique values for this key is much higher than 10, it
seems that the aggregation's scope is global as far as it searches for
documents with a value identical to one of the 10 listed, but it is limited
as far as the key values used in the aggregation is concerned.

How do I need to set up my curl command in order for the aggregation to
return more key-value pairs?

Many thanks,
Diederik

Command:
curl -XGET 'http://localhost:9200/_search?pretty=true' -d '{ "size": 0,
"aggs": { "datum_uitspraak_ymd": { "terms": { "field":
"datum_uitspraak_ymd" } } } }'

Returns:
"aggregations" : { "datum_uitspraak_ymd" : { "buckets" : [ { "key" :
"20121219", "doc_count" : 612 }, { "key" : "20120516", "doc_count" : 526 },
{ "key" : "20110601", "doc_count" : 472 }, { "key" : "20121218",
"doc_count" : 468 }, { "key" : "20090520", "doc_count" : 349 }, { "key" :
"20101222", "doc_count" : 274 }, { "key" : "20120711", "doc_count" : 272 },
{ "key" : "20090429", "doc_count" : 246 }, { "key" : "20120718",
"doc_count" : 230 }, { "key" : "20120425", "doc_count" : 226 } ] } }

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/wCkb_xHaUmQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/9f0cb2af-fa2d-40b0-9930-b884141f2969%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/9f0cb2af-fa2d-40b0-9930-b884141f2969%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/bcd2fd44-6771-4c30-acfc-68ad731074b3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

5

Awesome, just ran the curl command below, works fine!

curl -XGET 'http://localhost:9200/_search?pretty=true' -d '{ "size": 0, "aggs": { "datum_uitspraak_ymd": { "terms": { "field": "datum_uitspraak_ymd", "include":"2014.*", "size":10000 } } } }'

Op Jul 7, 2014, om 3:10 PM heeft Colin Goodheart-Smithe colin.goodheart-smithe@elasticsearch.com het volgende geschreven:

Glad it worked.

Yes, there are options for includes and excludes patterns. Take a look at the following link for information on how to use them.
Elasticsearch Platform — Find real-time answers at scale | Elastic

Colin

On Monday, 7 July 2014 14:04:56 UTC+1, Diederik Meijer wrote:
Hi Colin,

Thank you, that works perfectly!

Is there any way to limit the key-value pairs by a certain parameter, in the example below: to limit the aggregation to "datum_uitspraak_ymd" keys that start with "2014"?

Or does that require the combination of a filter and an aggregation?

Many thanks,

Diederik

Op Jul 7, 2014, om 2:47 PM heeft Colin Goodheart-Smithe colin.goodheart-smithe@elasticsearch.com het volgende geschreven:

Diederik,

To increase the number of terms returned by the terms aggregation you will need to add the 'size' parameter to your aggregation. The below curl command will return you the top 200 terms (ordered by decending doc_count).

curl -XGET 'http://localhost:9200/_search?pretty=true' -d '{ "size": 0, "aggs": { "datum_uitspraak_ymd": { "terms": { "field": "datum_uitspraak_ymd", "size" : 200 } } } }'

You may also find the following link to the documentation useful regarding the size parameter of the terms aggregation.

Elasticsearch Platform — Find real-time answers at scale | Elastic

Hope this helps,

Colin

On Monday, 7 July 2014 13:09:52 UTC+1, Diederik Meijer wrote:
Dear list,

I need to create an aggregation by a specific field, named "datum_uitspraak_ymd'. I am using the below curl command and it works fine in a sense that it returns the aggregation listed below. While this result seems OK enough, it seems that the keys listed in the aggregation are limited to those listed in 10 records.

As the number of unique values for this key is much higher than 10, it seems that the aggregation's scope is global as far as it searches for documents with a value identical to one of the 10 listed, but it is limited as far as the key values used in the aggregation is concerned.

How do I need to set up my curl command in order for the aggregation to return more key-value pairs?

Many thanks,
Diederik

Command:
curl -XGET 'http://localhost:9200/_search?pretty=true' -d '{ "size": 0, "aggs": { "datum_uitspraak_ymd": { "terms": { "field": "datum_uitspraak_ymd" } } } }'

Returns:
"aggregations" : { "datum_uitspraak_ymd" : { "buckets" : [ { "key" : "20121219", "doc_count" : 612 }, { "key" : "20120516", "doc_count" : 526 }, { "key" : "20110601", "doc_count" : 472 }, { "key" : "20121218", "doc_count" : 468 }, { "key" : "20090520", "doc_count" : 349 }, { "key" : "20101222", "doc_count" : 274 }, { "key" : "20120711", "doc_count" : 272 }, { "key" : "20090429", "doc_count" : 246 }, { "key" : "20120718", "doc_count" : 230 }, { "key" : "20120425", "doc_count" : 226 } ] } }

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/wCkb_xHaUmQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9f0cb2af-fa2d-40b0-9930-b884141f2969%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/wCkb_xHaUmQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/bcd2fd44-6771-4c30-acfc-68ad731074b3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/578C3BD6-8AC2-46E0-A6C7-EC3F51A44645%40tenhorses.com.
For more options, visit https://groups.google.com/d/optout.