How can I easily convert logstash filters with grok to ingest pipelines?

Elastic Stack Elasticsearch
1

Logstash has ingest-convert.sh to convert ingest pipelines to Logstash, but I'd like to convert the other way around.

I have the 23-Tomcat-filters and grok_patterns file.

Is there any smart way that would save me time writing all json files to this ingest pipeline-format, based on the current logstash configuration?

PUT _ingest/pipeline/ujjain
{
  "description" : "Pipeline ujjain",
  "processors": [
    {
      "grok": {
        "field": "message",
        "patterns": ["%{MONTH}%{SPACE}%{MONTHDAY},%{SPACE}%{YEAR}%{SPACE}%{HOUR}:?%{MINUTE}(?::?%{SECOND})%{SPACE}(?:AM|PM)%{SPACE}%{NOTSPACE:class}%{SPACE}%{NOTSPACE:type_log}%{SPACE}%{WORD:loglevel}:%{SPACE}%{GREEDYDATA:log_text}"]
    }

...

2

If you already have them in Logstash, why do you want to migrate back to ingest pipelines?

Ingest node was thought as an initial step to get log parsing working without the requirement of running Logstash if you are just doing simple parsing.

Hence, there is no real gain to migrate back to ingest pipelines if you already have Logstash working.

1 Like
3

This blog post discusses the differences in capabilities between Logstash and ingest nodes. Logstash has more functionality, so it would help to see your existing config to judge if it would be possible to implement this efficiently as ingest node pipelines or not.

1 Like
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.