Grok Processor Custom Patterns

I am trying to find out how to add new patterns to

GET _ingest/processor/grok

I would like to do this so I can use filebeat to send a few extra log types to elasticsearch. Currently I am sending a few things directly to elasticsearch and it looks like I can't specify certain logs to go to logstash instead.

What is the best practice for adding custom patterns that filebeat can use? I haven't had good luck with grok patterns that are defined inside modules. I can make a new instance of filebeat that runs and sends to logstash, and set it up as an additional service on the host, but was hoping to avoid this as I am not sure how well it will jive if the two filebeat instances are trying to use the same registry.

Any ideas / suggestions are much appreciated.

Are you searching how to add certain patterns within Elasticsearch? If so you can use the pattern definitions as part of the processor in the pipeline, see https://www.elastic.co/guide/en/elasticsearch/reference/7.0/grok-processor.html#custom-patterns (which also means, you have to specify per processor).

What I am trying to do specifically is add another timestamp pattern that isn't present in the grok processor list of available patterns. I am hoping there is a way to add a non-existant pattern to the Grok Processor in elasticsearch that lists when you run

_ingest/processor/grok

The time pattern that does not currently exist looks like this, at least if it exists I haven't been able to identify it.

[18-Apr-2019 17:14:51 America/Chicago]

%{MONTHDAY}-%{MONTH}-%{YEAR}....etc

I haven't had any luck specifying it inside of filebeat configurations

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.