Local Storage in Elastic/kibana/Logstash VM

Elastic Stack Elasticsearch
1

Hello Team,

Im concern about the storage utility in my Elastic/Kibana/Logstash instance, is only 4 months up and 40 fleet agents and already used 500gb, is a normal behaviour?

How can I avoid the high storage utility?

Thanks in advance.
Kind regards,

2

What are the integrations that you are using? The space used depends on the data that's been collected.

Please describe the integrations you are using in your agents.

3

Hello Leandro!

Thanks for the reply.

Find below my current integrations.

imagen
imagen2164×877 87.7 KB

Kind regards,

4

Yeah, but how are your policies organized?

You are running all those integrations in all agents? It does not make much sense.

Do you have one single policy for all agents or do you have multiple policies? It is expected to have multiple policies.

But from what you shared the amount of data seems to be ok, it is basically something around 100 MB per host per day, which is reasonable when you have things like metrics and an xdr (elastic defend) running, both can get a lot of logs.

5

Found the 2 index.

imagen
imagen1370×160 23.3 KB

I understand the system integration due it has 44 hosts, but my windows integration? Only have 7 hosts on it and it is 164Gb.

6

This is a metrics integration, metics can generate a lot of data and it also can vary from server to server.

You may have a noisy server with a lot of things happening that would result in a lot of metrics for the perfmon dataset.

The solution is to check the integration and see if you really need all the metrics that are getting collected.

7

Hello Leandro,

Thanks for the reply.
Will configure Index Lifecycles to avoid the high use of the storage in the VM.
If I configure to move the index to another VM, will can import again in case to check any log in the future?

Kind regards,