# Efficient way of storing data in index to support date range filter

**URL:** <https://discuss.elastic.co/t/efficient-way-of-storing-data-in-index-to-support-date-range-filter/140650>\
**Category:** Elasticsearch\
**Created:** [July 19, 2018, 5:13am UTC](https://discuss.elastic.co/t/efficient-way-of-storing-data-in-index-to-support-date-range-filter/140650 "2018-07-19T05:13:06Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![BleedCode](https://avatars.discourse-cdn.com/v4/letter/b/a87d85/32.png) [@BleedCode](https://discuss.elastic.co/u/BleedCode)\
**Post date:** [July 19, 2018, 5:13am UTC](https://discuss.elastic.co/t/efficient-way-of-storing-data-in-index-to-support-date-range-filter/140650/1 "2018-07-19T05:13:06Z")

</div>

Hi All,

I was looking for an efficient solution by using which i should be able to select any date range data. Example: if i want to show a performance report for 2 months or 3 days the performance of system should not be effected much. Currently, i am storing index on monthly basis but with this logic elasticsearch performance goes down as i have 5 nodes and 3 shards for each node. And ES searches for data in each 5\*3 nodes which is a time consuming process. Please suggest some design for this.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [July 19, 2018, 5:28am UTC](https://discuss.elastic.co/t/efficient-way-of-storing-data-in-index-to-support-date-range-filter/140650/2 "2018-07-19T05:28:07Z")

</div>

How much data do you have? What is the average shard size you are querying? What is the specification of your cluster, especially the hosts Elasticsearch is deployed on?

---

<div class="post-metadata">

**Author:** ![BleedCode](https://avatars.discourse-cdn.com/v4/letter/b/a87d85/32.png) [@BleedCode](https://discuss.elastic.co/u/BleedCode)\
**Post date:** [July 19, 2018, 5:31am UTC](https://discuss.elastic.co/t/efficient-way-of-storing-data-in-index-to-support-date-range-filter/140650/3 "2018-07-19T05:31:03Z")

</div>

It's is a real time data so it will keep on increasing.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [July 19, 2018, 5:32am UTC](https://discuss.elastic.co/t/efficient-way-of-storing-data-in-index-to-support-date-range-filter/140650/4 "2018-07-19T05:32:30Z")

</div>

That does not really answer any of my questions. have you looked at what system resources are milting performance when you are querying? Is it CPU? Is it disk I/O and iowait? Are you seeing a lot go GC indicating it may be memory?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 16, 2018, 5:32am UTC](https://discuss.elastic.co/t/efficient-way-of-storing-data-in-index-to-support-date-range-filter/140650/5 "2018-08-16T05:32:31Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.