Index size growth


(Fiona) #1

I am trying to figure out how best to plan for disk space/EBS volume size for AWS ES.

When I get the weekly index details back like this:

|green|open|filebeat-test-2018.42|hpJXziswRh-iGY1FOZnqNw|5|1|1309774|0|2.3gb|1.1gb
|green|open|filebeat-test-2018.43|zV5NVuiaTA2IotJlervl7Q|5|1|56541495|0|50.4gb|25.2gb
|green|open|filebeat-test-2018.44|EH6i4OhDQi-bOtw1ctGxYA|5|1|187165913|0|168.2gb|84.1gb

Does this mean week .44 has 2.3GB + 25.2GB + new 116GB = for total 168GB?
So weekly data growth for this index is 2GB, then 48GB and then 116GB?

I am trying to understand because I have another index where week .43 is less than the other 2 weeks and I don't understand how that could be!

Fiona


(Gordon Brown) #2

Each row is a separate index, and each index's size here reflects only the data in that week, not a cumulative sum.

You have three indices:

  1. filebeat-test-2018.42 that has approximately 1.1gb of data with one replica, for a total of 2.3gb in that index.
  2. filebeat-test-2018.32 that has approximately 25.2gb of data with one replica, for a total of 50.4gb in that index.
  3. filebeat-test-2018.43 that has approximately 84.1gb of data with one replica, for a total of 168.2gb in that index.

So the total amount of storage being consumed on this by these three indices, is 2.3gb + 50.4gb + 168.2gb = 220.9gb.

The size of each index just depends on how much log data was produced that week.

Does that help at all?

As an aside, I recommend Elastic's Elasticsearch Service for running Elasticsearch in AWS .


(Fiona) #3

Yes, this was very helpful Gordon, thanks for the clarification!


(system) #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.