# Elasticsearch aggregations for analytics

**URL:** <https://discuss.elastic.co/t/elasticsearch-aggregations-for-analytics/22229>\
**Category:** Elasticsearch\
**Created:** [February 18, 2015, 9:17am UTC](https://discuss.elastic.co/t/elasticsearch-aggregations-for-analytics/22229 "2015-02-18T09:17:42Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Itai\_Yaffe](https://avatars.discourse-cdn.com/v4/letter/i/41988e/32.png) [@Itai\_Yaffe](https://discuss.elastic.co/u/Itai_Yaffe)\
**Post date:** [February 18, 2015, 9:17am UTC](https://discuss.elastic.co/t/elasticsearch-aggregations-for-analytics/22229/1 "2015-02-18T09:17:42Z")

</div>

Hey,  
We have an Elasticsearch cluster in production with 20 nodes which hold a  
few TBs of data and loading millions of documents a day.  
We use Elasticsearch for analytics purposes and the main thing we're  
interested is counting unique users.

We started the production search with Elasticsearch 0.9.X, when there was  
no cardinality aggregation, therefor we were bound to create the document  
structure as seen below.  
Most of our queries are looking for the unique users count based on a date  
range and specific segments.  
Some of our analytic UI screens require executing hundreds of queries in  
parallel and one even requires thousands of queries.

When migrating to V1.4, we hoped to start using the aggregation feature,  
but even with the doc\_values enabled, we experience aggregation time of  
_minutes_...  
We're running on c3.8xlarge EC2 instances with 60GB RAM, of which 30GB are  
allocated to ES heap.  
We have 6 indexes with 2 replicas each, each index has 20 shards.  
Each aggregation/query is performed against a single index (see aggregation  
example below).

Has anyone dealt with such use cases?

Thanks!

_Document structure_ :  
{  
"user": {  
"\_ttl": {  
"enabled": true  
},  
"properties": {  
"events": {  
"type": "nested",  
"properties": {  
"event\_time": {  
"type": "date",  
"format": "dateOptionalTime",  
"doc\_values" : true  
},  
"segments": {  
"properties": {  
"segment": {  
"type": "string",  
"index": "not\_analyzed",  
"doc\_values" : true  
}  
}  
}  
}  
}  
}  
}  
}

For example :  
{  
"\_index": "...",  
"\_type": "...",  
"\_id": "...",  
"\_version": 1,  
"\_score": 1,  
"\_source": {  
"events": [  
{  
"event\_time": "2014-11-03",  
"segments": [  
{  
"segment": "ALICE"  
},  
{  
"segment": "BOB"  
}  
]  
},  
{  
"event\_time": "2014-11-04",  
"segments": [  
{  
"segment": "RON"  
},  
{  
"segment": "YULA"  
}  
]  
}  
]  
}  
}

_Aggegation example_ :  
{  
"size": 0,  
"query": {  
"nested": {  
"query": {  
"filtered": {  
"query": {  
"match\_all": {}  
},  
"filter": {  
"bool": {  
"must": [  
{  
"range": {  
"events.event\_time": {  
"from": "2014-11-17",  
"to": "2014-11-24",  
"include\_lower": true,  
"include\_upper": true  
}  
}  
}  
]  
}  
}  
}  
},  
"path": "events"  
}  
},  
"aggregations": {  
"nested": {  
"nested": {  
"path": "events"  
},  
"aggregations": {  
"segments": {  
"terms": {  
"field": "events.segments.segment",  
"size": 0  
},  
"aggregations": {  
"uu": {  
"reverse\_nested": {}  
}  
}  
}  
}  
}  
}  
}

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/acbc3022-8845-4170-999d-d0b2bc9dfeb3%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/acbc3022-8845-4170-999d-d0b2bc9dfeb3%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 12:31am UTC](https://discuss.elastic.co/t/elasticsearch-aggregations-for-analytics/22229/2 "2017-07-06T00:31:58Z")

</div>

Closed on July 6, 2017, 12:31am UTC