Elastic ingest pipeline grok filter error

Elastic Stack Kibana
1

hi,
I am trying to to build an ingest pipeline using grok processor
my grok filter works on the Kibana->console .

grok filter

^%{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{TIME}.%{LOGLEVEL:log-level} : \[%{DATA:user}] \[%{DATA:correlation-id}] 

2020-11-12 19:01:26.999 INFO : [http-nio-9090-exec-34] [xyz.b227ecf2-b770-468e-bb27-3a2c9e3d97b0]

but when i use the same one in

POST _ingest/pipeline/_simulate
{
  "pipeline": {
  "description" : "parse multiple patterns",
  "processors": [
    {
      "grok": {
        "field": "message",
        "patterns": [" ^%{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{TIME}.%{LOGLEVEL:log-level} : \[%{DATA:user}] \[%{DATA:correlation-id}]"]
      }
    }
  ]
},
"docs":[
  {
    "_source": {
      "message": "2020-11-12 19:01:26.999 INFO : [http-nio-9090-exec-34] [xyz.b227ecf2-b770-468e-bb27-3a2c9e3d97b0]"
    }
  }
  ]
}

The problem seems to be \ in the grok filter. Not sure why.
Appreciate any suggestions or pointers.

Thanks

2

Welcome @vsv0001

Give this a go.

POST _ingest/pipeline/_simulate
{
  "pipeline": {
  "description" : "parse multiple patterns",
  "processors": [
    {
      "grok": {
        "field": "message",
        "patterns": ["^%{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{TIME}.%{LOGLEVEL:log-level} : \\[%{DATA:user}\\] \\[%{DATA:correlation-id}\\]"]
      }
    }
  ]
},
"docs":[
  {
    "_source": {
      "message": "2020-11-12 19:01:26.999 INFO : [http-nio-9090-exec-34] [xyz.b227ecf2-b770-468e-bb27-3a2c9e3d97b0]"
    }
  }
  ]
}

Or you can also do it like this.

"patterns": [
            """^%{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{TIME}.%{LOGLEVEL:log-level} : \[%{DATA:user}\] \[%{DATA:correlation-id}\]"""
          ]
3

Thank you. :smiley:

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.