Bulk Payload Sizes


#1

Hi,
Is there a way that I can track the size of the bulk ingestion payloads hitting elasticsearch?

Regards,
D


(Thiago Souza) #2

You can use the Mapper Size Plugin that will add a _size metadata field with the raw size of the json document.


#3

I probably didn't make myself clear enough. That will give the size of the message payload. i want to track how many requests are in each bulk indexing request sent by logstash.


(Thiago Souza) #4

Unfortunately no metric will tell you that exactly.


#5

Is that likely to change?


(Thiago Souza) #6

There is nothing in the roadmap. What is generally required to know is the actual indexing rate as docs per second. May I ask why you want to keep track of numbers of documents per bulk?


#7

So I can gauge the efficiency of the logstash to elasticsearch interface. Could I get this information by parsing the output of the task api?


(Thiago Souza) #8

You can use Logstash Monitoring API to calculate throughput at plugin level.

X-Pack (with Basic license) will also show these values in Pipeline Viewer UI


(system) #9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.