Hi, I am provisioning the ELK stack on AWS kubernetes (EKS) with EBS storage.
Can someone help me in configuring the IOPS?
How much IOPS do I need to configure for EBS io2, for elastic stack?
This really depends strongly on how much work you'll be doing. Querying will also fetch docs from disk, though document writing, updating and deleting will have an even bigger impact. Honestly you just have to try it - either simulate a workload or put some real workload on the cluster and see if it falls over. Note when it falls over (= becomes too slow for your needs) and correlate that moment with a disk I/O graph.
If you were not on AWS EKS I'd say use our Metricbeat to monitor disk I/O and discern the right I/O level. I don't know if you can use it on EKS, but here are the instructions for Metricbeat-on-K8S (scroll to "Get started in minutes" and select "Kubernetes Metrics").
You can also just save yourself all that trouble by using Elastic Cloud. We have taken care of configuring IOps and a number of config profiles are available (shown below).
You don't even need to move anything out of AWS. If you don't like the standalone Elastic Cloud for billing or data governance reasons, just use it via the AWS Marketplace instead. Here's a blog post that can help if you need to go this way.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.