Grok Processor Custom Patterns

[Ryne_Keel]

I am trying to find out how to add new patterns to

GET _ingest/processor/grok

I would like to do this so I can use filebeat to send a few extra log types to elasticsearch. Currently I am sending a few things directly to elasticsearch and it looks like I can't specify certain logs to go to logstash instead.

What is the best practice for adding custom patterns that filebeat can use? I haven't had good luck with grok patterns that are defined inside modules. I can make a new instance of filebeat that runs and sends to logstash, and set it up as an additional service on the host, but was hoping to avoid this as I am not sure how well it will jive if the two filebeat instances are trying to use the same registry.

Any ideas / suggestions are much appreciated.

[spinscale (Alexander Reelsen)]

Are you searching how to add certain patterns within Elasticsearch? If so you can use the pattern definitions as part of the processor in the pipeline, see https://www.elastic.co/guide/en/elasticsearch/reference/7.0/grok-processor.html#custom-patterns (which also means, you have to specify per processor).

[Ryne_Keel]

What I am trying to do specifically is add another timestamp pattern that isn't present in the grok processor list of available patterns. I am hoping there is a way to add a non-existant pattern to the Grok Processor in elasticsearch that lists when you run

_ingest/processor/grok

The time pattern that does not currently exist looks like this, at least if it exists I haven't been able to identify it.

[18-Apr-2019 17:14:51 America/Chicago]

%{MONTHDAY}-%{MONTH}-%{YEAR}....etc

I haven't had any luck specifying it inside of filebeat configurations