# Kafka\_consumer\_lag in Logstash

**URL:** <https://discuss.elastic.co/t/kafka-consumer-lag-in-logstash/324108>\
**Category:** Logstash\
**Created:** [January 27, 2023, 12:41pm UTC](https://discuss.elastic.co/t/kafka-consumer-lag-in-logstash/324108 "2023-01-27T12:41:33Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ondrej\_S](https://avatars.discourse-cdn.com/v4/letter/o/dfb087/32.png) [@Ondrej\_S](https://discuss.elastic.co/u/Ondrej_S)\
**Post date:** [January 27, 2023, 12:41pm UTC](https://discuss.elastic.co/t/kafka-consumer-lag-in-logstash/324108/1 "2023-01-27T12:41:33Z")

</div>

Hello,

Is it possible to get the value or verify value of “kafka\_consumer\_lag” directly in Logstash 7.17?

The idea is to verify:  
`if [kafka_consumer_lag][lag] == 0`

In Logstash Output and if yes run http plugin with  
`url => http://localhost:5601/_enrich/policy/policy_name/_execute`

To execute enrich policy every time source data are updated, instead of manual execution which is needed by the default Elastic design. So, the enrich processor pipeline automatically updates the final index with fresh data from the source.

Or is there another way how to automate enrich policy execution?

I know about [Enrich processor: allow scheduling of policy executions · Issue #50071 · elastic/elasticsearch · GitHub](https://github.com/elastic/elasticsearch/issues/50071) “Enrich processor: allow scheduling of policy executions”, which is still not resolved.

Thank you

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [January 27, 2023, 12:57pm UTC](https://discuss.elastic.co/t/kafka-consumer-lag-in-logstash/324108/2 "2023-01-27T12:57:52Z")

</div>

> [@Ondrej\_S](#):
>
> Is it possible to get the value or verify value of “kafka\_consumer\_lag” directly in Logstash 7.17?

No, it is not possible.

> [@Ondrej\_S](#):
>
> Or is there another way how to automate enrich policy execution?

The easiest way is to have a crontab to run the `_execute`, this is what I did in my case, every couple of hours I make a request to `_execut` the policy and update the enrich index if there are any changes.

---

<div class="post-metadata">

**Author:** ![Ondrej\_S](https://avatars.discourse-cdn.com/v4/letter/o/dfb087/32.png) [@Ondrej\_S](https://discuss.elastic.co/u/Ondrej_S)\
**Post date:** [January 30, 2023, 2:28pm UTC](https://discuss.elastic.co/t/kafka-consumer-lag-in-logstash/324108/3 "2023-01-30T14:28:23Z")

</div>

Thank you for you reply @leandrojmp  
Could you pls show me how did you set up the crontab?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 27, 2023, 2:29pm UTC](https://discuss.elastic.co/t/kafka-consumer-lag-in-logstash/324108/4 "2023-02-27T14:29:18Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.