# Standard rules for ELK handling big data?

**URL:** <https://discuss.elastic.co/t/standard-rules-for-elk-handling-big-data/108540>\
**Category:** Elasticsearch\
**Created:** [November 21, 2017, 9:40am UTC](https://discuss.elastic.co/t/standard-rules-for-elk-handling-big-data/108540 "2017-11-21T09:40:31Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Gobinda\_Nandi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gobinda_nandi/32/24506_2.png) [@Gobinda\_Nandi](https://discuss.elastic.co/u/Gobinda_Nandi)\
**Post date:** [November 21, 2017, 9:40am UTC](https://discuss.elastic.co/t/standard-rules-for-elk-handling-big-data/108540/1 "2017-11-21T09:40:31Z")

</div>

what are the standard rules for using ELK handling big ( millions ) data?

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [November 21, 2017, 10:33am UTC](https://discuss.elastic.co/t/standard-rules-for-elk-handling-big-data/108540/2 "2017-11-21T10:33:57Z")

</div>

> [@Gobinda\_Nandi](#):
>
> handling big ( millions ) data?

Millions may well still be a single machine.

Billions uses multiple machines, splitting a single index into `shards`.  
A never ending-stream of new content is managed using multiple time-based indices - one per day or week perhaps and deleting old indices beyond the retention period. The retention period is whatever is legal or what you decide is the crossover between potentially interesting content and the economic viability of keeping it around.

---

<div class="post-metadata">

**Author:** ![Gobinda\_Nandi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gobinda_nandi/32/24506_2.png) [@Gobinda\_Nandi](https://discuss.elastic.co/u/Gobinda_Nandi)\
**Post date:** [November 21, 2017, 10:54am UTC](https://discuss.elastic.co/t/standard-rules-for-elk-handling-big-data/108540/3 "2017-11-21T10:54:23Z")

</div>

thanks man !!!!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 19, 2017, 10:54am UTC](https://discuss.elastic.co/t/standard-rules-for-elk-handling-big-data/108540/4 "2017-12-19T10:54:46Z")

</div>

Closed on December 19, 2017, 10:54am UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.