# Ingest Node Processor for cumulative count of records

**URL:** <https://discuss.elastic.co/t/ingest-node-processor-for-cumulative-count-of-records/108974>\
**Category:** Elasticsearch\
**Created:** [November 24, 2017, 6:49am UTC](https://discuss.elastic.co/t/ingest-node-processor-for-cumulative-count-of-records/108974 "2017-11-24T06:49:50Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![RohanKumbhar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rohankumbhar/32/23649_2.png) [@RohanKumbhar](https://discuss.elastic.co/u/RohanKumbhar)\
**Post date:** [November 24, 2017, 6:49am UTC](https://discuss.elastic.co/t/ingest-node-processor-for-cumulative-count-of-records/108974/1 "2017-11-24T06:49:51Z")

</div>

Hi Team,

Is there any processor in ingest pipeline which will give me the cumulative count the records processed through the ingest node?

for example there are 10 records processed by ingest node then cumulative count should be 10 and for next record it should be 11.

Thanks,  
Rohan

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [November 24, 2017, 7:07am UTC](https://discuss.elastic.co/t/ingest-node-processor-for-cumulative-count-of-records/108974/2 "2017-11-24T07:07:29Z")

</div>

No. It doesn’t exist and I don’t think it can exist TBH.

---

<div class="post-metadata">

**Author:** ![RohanKumbhar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rohankumbhar/32/23649_2.png) [@RohanKumbhar](https://discuss.elastic.co/u/RohanKumbhar)\
**Post date:** [November 24, 2017, 7:52am UTC](https://discuss.elastic.co/t/ingest-node-processor-for-cumulative-count-of-records/108974/3 "2017-11-24T07:52:27Z")

</div>

Thanks @dadoonet for your response!! Just a suggestion; can we add processor "metrics processor" similar to metrics filter plugin of logstash ? which help to get the required details.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [November 24, 2017, 8:08am UTC](https://discuss.elastic.co/t/ingest-node-processor-for-cumulative-count-of-records/108974/4 "2017-11-24T08:08:44Z")

</div>

I don't know how the metrics filter plugin works behind the scene but I believe it is storing "locally" on the Logstash instance the "counters". Which means to me that it can not be distributed within multiple Logstash instances.

With that said, ingest node is distributed by nature. Which means that Node 1 will handle a 1st request, then may be Node 2, then may be Node3... And you can easily understand what is going to happen in that case.

If Logstash could work for you, I'd go the logstash way.  
If you really want to do it in Ingest, then you need to:

- write your own ingest plugin
- make sure that only one node is set to `"node.ingest": true` and other nodes are set to false.

My 2 cents.

---

<div class="post-metadata">

**Author:** ![RohanKumbhar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rohankumbhar/32/23649_2.png) [@RohanKumbhar](https://discuss.elastic.co/u/RohanKumbhar)\
**Post date:** [November 24, 2017, 2:41pm UTC](https://discuss.elastic.co/t/ingest-node-processor-for-cumulative-count-of-records/108974/5 "2017-11-24T14:41:15Z")

</div>

sure Thanks @dadoonet !!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 22, 2017, 2:42pm UTC](https://discuss.elastic.co/t/ingest-node-processor-for-cumulative-count-of-records/108974/6 "2017-12-22T14:42:05Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.