How to get size (total) of events send by a host per day

Elastic Stack Elasticsearch
1

We have been asked different scenarios

  1. What is the original size/volume of data send by a particular host?
  2. What is the size/volume of data indexed into ES from a particular host

For (1) it is actually the size of the "message"
My thought was to calculate size per message and then multiply by number of messages per day from the host

Is my understanding correct that the "message" field which comes into ES (if there was no alteration) is the original _raw data from the client system?

2

I'd may be use the mapper size plugin.

That would give you a raw idea of the size of every document.
Then if you filter by host and run a sum agg to the size field, that'd give you an idea of the size of the related documents.

1 Like
3

thanks for that. may be not approved in our installation. Any other methods to find the size of _source field of already indexed fields?

I tried scrripts (using message.keyword, length) but all in vain

4

how about you do estimate. this is what I do.

run ingestion for a day from A system. after 24 hour I stop that.

go to index management and check the size and document count.
size/count = your single document size.

1 Like
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.