Drop Field - Meta data value

[yasar (Yasar Arafaath A)]

Hi team,

We would like to know more on drop fields from Filebeat and Metricbeat data,
As we implemented in Dev environment and we able to see the dropped field in dev Kibana. so we need to implement in same as in test environment.

We are able to see the logs sizes when we pushing the data of 400 mb by manually and getting the data size in Index Management like 1.3 GB and after dropping (Agent Id, Agent Type, Agent Name nd Agent version, add host meta data and add cloud metadata etc ...,) these fields in filebeat we got that 1.1 GB of data in Index Management. So, We could seen the reduced sizes after dropping the fields.

Before Dropping in Filebeat - 1.3 GB
After Dropping in Filebeat - 1.1 GB

We have dropping the fields based on unnecessary fields which it is allocated the spaces. So, our main concern is that, we need to find out the values for host metadata and cloud metadata. Kindly find the attached screenshot for you reference in filebeat.yml

Could you please guide us, how to find the data sizes for host metadata and cloud metadata.

Thanks & regards,
Yasar Arafaath A.

[yasar (Yasar Arafaath A)]

Hi team,

Could you please update on the mentioned issue.

Thanks & Regards,
Yasar Arafaath A

[mtojek (Marcin Tojek)]

Did you try to throw away host metadata and see results? You can prepare another index for performance measurements.

[yasar (Yasar Arafaath A)]

Hi Marcin,

Thanks for your reply.

If we need to take the metadata size without dropping the fields. That how could we take that data.

Thanks,
Yasar Arafaath A.