# Low latency multi dimensional aggregation of varying dims number/order

**URL:** <https://discuss.elastic.co/t/low-latency-multi-dimensional-aggregation-of-varying-dims-number-order/64906>\
**Category:** Elasticsearch\
**Created:** [November 3, 2016, 4:18pm UTC](https://discuss.elastic.co/t/low-latency-multi-dimensional-aggregation-of-varying-dims-number-order/64906 "2016-11-03T16:18:08Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![shadim](https://avatars.discourse-cdn.com/v4/letter/s/b487fb/32.png) [@shadim](https://discuss.elastic.co/u/shadim)\
**Post date:** [November 3, 2016, 4:18pm UTC](https://discuss.elastic.co/t/low-latency-multi-dimensional-aggregation-of-varying-dims-number-order/64906/1 "2016-11-03T16:18:08Z")

</div>

Hello,

I would like to know if ES can fit our use case or not before investing into a POC. Basically, we need to execute aaggregated queries (sum/avg) over metrics in impressions/clicks logs given size is in TBs and a varying number/order of dimensions (up to 20) in addition a response time of less than 3 seconds.

I understand that this is a easy job for Vertica but i would like to know if ES can compete in this area as well. If yes, what is usually the typical cluster size?

Thank you

---

<div class="post-metadata">

**Author:** ![shadim](https://avatars.discourse-cdn.com/v4/letter/s/b487fb/32.png) [@shadim](https://discuss.elastic.co/u/shadim)\
**Post date:** [November 6, 2016, 8:54am UTC](https://discuss.elastic.co/t/low-latency-multi-dimensional-aggregation-of-varying-dims-number-order/64906/2 "2016-11-06T08:54:32Z")

</div>

Does anybody have a clue?

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [November 7, 2016, 8:39am UTC](https://discuss.elastic.co/t/low-latency-multi-dimensional-aggregation-of-varying-dims-number-order/64906/3 "2016-11-07T08:39:08Z")

</div>

Hey,

this question is very generic and does not contain too much concrete requirement in terms of how your data looks like, how your query look like, hardware, etc. I think it would be much easier to just setup Elasticsearch, index data, query it and see if you are happy with the response - or ask further once you have a more concrete prototype up and running and can provide better insights in what you are doing - or get commercial help from Elastic for this.. \</sales\> 🙂

--Alex

---

<div class="post-metadata">

**Author:** ![shadim](https://avatars.discourse-cdn.com/v4/letter/s/b487fb/32.png) [@shadim](https://discuss.elastic.co/u/shadim)\
**Post date:** [November 9, 2016, 2:58pm UTC](https://discuss.elastic.co/t/low-latency-multi-dimensional-aggregation-of-varying-dims-number-order/64906/4 "2016-11-09T14:58:28Z")

</div>

Well, I took your advice and setup a single ES 5.x node on AWS with the following specs:  
16 Core, 122GB RAM, and SSD (3000 IOPS)

I imported 600M+ documents into ES using logstash, and i executed an ES query that looks like:  
SELECT Id, SUM(x), SUM(y), SUM(z) GROUP BY ID LIMIT 10

The query took ~28 seconds.

Is this something usual? or i am missing something here?

-SM

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 10:06pm UTC](https://discuss.elastic.co/t/low-latency-multi-dimensional-aggregation-of-varying-dims-number-order/64906/5 "2017-07-05T22:06:01Z")

</div>


