# Bulk Payload Sizes

**URL:** <https://discuss.elastic.co/t/bulk-payload-sizes/134762>\
**Category:** Elasticsearch\
**Created:** [June 6, 2018, 8:52am UTC](https://discuss.elastic.co/t/bulk-payload-sizes/134762 "2018-06-06T08:52:03Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![dawiro](https://avatars.discourse-cdn.com/v4/letter/d/71e660/32.png) [@dawiro](https://discuss.elastic.co/u/dawiro)\
**Post date:** [June 6, 2018, 8:52am UTC](https://discuss.elastic.co/t/bulk-payload-sizes/134762/1 "2018-06-06T08:52:03Z")

</div>

Hi,  
Is there a way that I can track the size of the bulk ingestion payloads hitting elasticsearch?

Regards,  
D

---

<div class="post-metadata">

**Author:** ![thiago](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thiago/32/32096_2.png) [@thiago](https://discuss.elastic.co/u/thiago)\
**Post date:** [June 6, 2018, 4:52pm UTC](https://discuss.elastic.co/t/bulk-payload-sizes/134762/2 "2018-06-06T16:52:56Z")

</div>

You can use the [Mapper Size Plugin](https://www.elastic.co/guide/en/elasticsearch/plugins/current/mapper-size.html) that will add a `_size` metadata field with the raw size of the json document.

---

<div class="post-metadata">

**Author:** ![dawiro](https://avatars.discourse-cdn.com/v4/letter/d/71e660/32.png) [@dawiro](https://discuss.elastic.co/u/dawiro)\
**Post date:** [June 11, 2018, 10:43am UTC](https://discuss.elastic.co/t/bulk-payload-sizes/134762/3 "2018-06-11T10:43:26Z")

</div>

I probably didn't make myself clear enough. That will give the size of the message payload. i want to track how many requests are in each bulk indexing request sent by logstash.

---

<div class="post-metadata">

**Author:** ![thiago](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thiago/32/32096_2.png) [@thiago](https://discuss.elastic.co/u/thiago)\
**Post date:** [June 11, 2018, 11:08am UTC](https://discuss.elastic.co/t/bulk-payload-sizes/134762/4 "2018-06-11T11:08:28Z")

</div>

Unfortunately no metric will tell you that exactly.

---

<div class="post-metadata">

**Author:** ![dawiro](https://avatars.discourse-cdn.com/v4/letter/d/71e660/32.png) [@dawiro](https://discuss.elastic.co/u/dawiro)\
**Post date:** [June 11, 2018, 11:09am UTC](https://discuss.elastic.co/t/bulk-payload-sizes/134762/5 "2018-06-11T11:09:54Z")

</div>

Is that likely to change?

---

<div class="post-metadata">

**Author:** ![thiago](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thiago/32/32096_2.png) [@thiago](https://discuss.elastic.co/u/thiago)\
**Post date:** [June 11, 2018, 11:23am UTC](https://discuss.elastic.co/t/bulk-payload-sizes/134762/6 "2018-06-11T11:23:10Z")

</div>

There is nothing in the roadmap. What is generally required to know is the actual indexing rate as docs per second. May I ask why you want to keep track of numbers of documents per bulk?

---

<div class="post-metadata">

**Author:** ![dawiro](https://avatars.discourse-cdn.com/v4/letter/d/71e660/32.png) [@dawiro](https://discuss.elastic.co/u/dawiro)\
**Post date:** [June 11, 2018, 2:47pm UTC](https://discuss.elastic.co/t/bulk-payload-sizes/134762/7 "2018-06-11T14:47:49Z")

</div>

So I can gauge the efficiency of the logstash to elasticsearch interface. Could I get this information by parsing the output of the task api?

---

<div class="post-metadata">

**Author:** ![thiago](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thiago/32/32096_2.png) [@thiago](https://discuss.elastic.co/u/thiago)\
**Post date:** [June 11, 2018, 3:16pm UTC](https://discuss.elastic.co/t/bulk-payload-sizes/134762/8 "2018-06-11T15:16:47Z")

</div>

You can use [Logstash Monitoring API](https://www.elastic.co/guide/en/logstash/current/node-stats-api.html#pipeline-stats) to calculate throughput at plugin level.

X-Pack (with Basic license) will also show these values in [Pipeline Viewer UI](https://www.elastic.co/guide/en/logstash/6.x/logstash-pipeline-viewer.html)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 9, 2018, 3:16pm UTC](https://discuss.elastic.co/t/bulk-payload-sizes/134762/9 "2018-07-09T15:16:48Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
