Efficient way of storing data in index to support date range filter

Elastic Stack Elasticsearch
1

Hi All,

I was looking for an efficient solution by using which i should be able to select any date range data. Example: if i want to show a performance report for 2 months or 3 days the performance of system should not be effected much. Currently, i am storing index on monthly basis but with this logic elasticsearch performance goes down as i have 5 nodes and 3 shards for each node. And ES searches for data in each 5*3 nodes which is a time consuming process. Please suggest some design for this.

2

How much data do you have? What is the average shard size you are querying? What is the specification of your cluster, especially the hosts Elasticsearch is deployed on?

3

It's is a real time data so it will keep on increasing.

4

That does not really answer any of my questions. have you looked at what system resources are milting performance when you are querying? Is it CPU? Is it disk I/O and iowait? Are you seeing a lot go GC indicating it may be memory?

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.