Conditions in Ingest node pipeline

Elastic Stack Elasticsearch
1

Hi,
I want to know if its possible to apply conditions in ingest node pipelines. I need to apply grok processor based on the value of one particular field. So, I will apply grok processor to extract that field and then make a check on that field. Depending on the result, I will apply one of two grok patterns to the rest of the event.

Thanks.

2

Conditionals are not supported. Better doing that in Logstash for that kind of advanced use case IMO.

3

Thank you for your response. Just wanted to check, I found this link:

This is a repository for a processor called "conditional". However, I am not able to use it as of course I have not installed it. Is this a legit processor or is it not usable at the moment?

4

A decision was made to not include this in the Ingest node due to the complexity it would introduce. The Ingest node is for simpler pipelines.

As mentioned, if you need conditionals in your pipeline, you should be investigating Logstash.

5

Thanks again for the prompt response. I actually have already implemented it using Logstash configuration but it is the client's request to implement it using ingest node pipelines since carrying out advanced filtration in the logstash configuration leads to excess CPU consumption, which they want to minimize.

6

What leads you to believe it would be any different in an ingest pipeline? Advanced filtering will inevitably lead to higher CPU usage. If you need that level of filtering, plan for the needed CPU utilization and build your stack accordingly.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.