"The processor action grok does not exist" in FileBeat. Why ? (Custom Logs Integration)

Elastic Stack Elasticsearch
1

I am a bit confused here.
Grok is available in Ingest Processors but not in Filebeat processors
May I ask why ? :thinking:

I was hoping to do this

Screenshot 2023-05-07 at 11.13.35
Screenshot 2023-05-07 at 11.13.35826×1074 72.3 KB

, but it breaks, as I am getting the following error :

[elastic_agent][error] [...] the processor action grok does not exist.

So,
1 - Why are the 2 processor families different ? why have two sets of processors and not just one ? (philosophical question)
2 - Where should I put my grok then ? (I am using custom logs integration)
My guess is that I need to define an ingest pipeline and add it to custom configurations.
Am I right ?

Thanks in advance,
M.

2

I think because ingest processor is an ingest pipeline, running on an elasticsearch node. FIlebeat processors are lightweight, limited and run on filebeat. Filebeat processors are good for dropping data before it traverses the network, but could put a lot of load on the agent if not kept simple.

3

Yep that's pretty much the reasoning behind the separation!

4

Ok, makes sense.
Thanks for the clarification :ok_hand:t5:

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.