# Hourly to Daily Aggregations

**URL:** <https://discuss.elastic.co/t/hourly-to-daily-aggregations/58662>\
**Category:** Elasticsearch\
**Created:** [August 23, 2016, 9:08am UTC](https://discuss.elastic.co/t/hourly-to-daily-aggregations/58662 "2016-08-23T09:08:15Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [August 23, 2016, 9:24am UTC](https://discuss.elastic.co/t/hourly-to-daily-aggregations/58662/2 "2016-08-23T09:24:11Z")

</div>

> [@alexkru](#):
>
> Can you please advice me with the best practices for the proccess/not to overload ES?

The 2 approaches for rolling up accurately would be as follows:

1. Examine all the data in your client code
2. Use the aggs framework to summarise the data for you.

Option 1 involves using the scan/scroll API to stream the data sorted by your chosen summary dimensions (e.g. hour/website) and reducing in your client code before writing using the bulk API to a new index (see [1] )  
Option 2 involves you repeatedly calling the aggs framework for a subset of the data e.g.

```
for all websites:
    aggs call to get daily stats for website

```

This can be a lot of calls if your grouping field is high cardinality so one way of breaking it up into a smaller set of single requests is to adopt the hash/modulo approach outlined here [2]

Cheers  
Mark

[1] [https://www.elastic.co/elasticon/2015/sf/building-entity-centric-indexes](https://www.elastic.co/elasticon/2015/sf/building-entity-centric-indexes) and [http://bit.ly/entcent](http://bit.ly/entcent)  
[2] [Getting accurate cardinality for a field in single shard index](https://discuss.elastic.co/t/getting-accurate-cardinality-for-a-field-in-single-shard-index/57302)

---

_[View the full topic](https://discuss.elastic.co/t/hourly-to-daily-aggregations/58662)._