EFK stack performance with large data

Elastic Stack Kibana
1

Hello Team,

We are planning to store logs in ElasticSearch DB through Filebeat and then create dashboards on Kibana for graphical visualization(Lens, TSVB, Aggregations) using basic license. As we are expecting around 25 million transactions per day so would there be any performance issues that we can expect? If yes, then how we can overcome those?

Thanks,

2

It really depends on the size of your data and the specs of your cluster, but 25 million events per day is not that much.

3

@leandrojmp thanks for the response.

So allocating enough memory to the box will not impact the performance? I am more concerned about the Kibana graphs response time.

If i have created a graph e.g TSVB with multiple aggregations and it need to show result by querying over 100 million transactions then how fast/slow would be the response time?

Thanks.

4

Kibana is the visualization and administration tool, the majority of response time depends on how fast your elasticsearch cluster can answer to Kibana requests.

This is something that you can't know without doing some tests as this depends on how your data looks like, what will be aggregations you need, if you mapped your index correctly and some other factors.

You need to start indexing your data and make some tests to know.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.