Generate grok custom pattern to message filed

Elastic Stack Elasticsearch
1

Hi All, I am having a query related to grok processor.
For example this is my message filed

{
"message":"agentId:agent003"
}

I want to Grok this and my output should me something like this

{
"message":"agentId:agent003",
"agentId":"agent003"
}

Could some one help me on this how to achieve this? If i am able to do it for one field i can manage for rest of my fields. Thanks in advance.

2

Try this:

\"agentId:%{NOTSPACE:agentId}\"

You can check at https://grokconstructor.appspot.com/do/match

3

Thanks a lot @fadjar340

4

Hi @fadjar340,

when i try to create a ingest pipeline using grok processor. Above mentioned grok pattern not accepted. Kindly advise.

    PUT _ingest/pipeline/dissectpipeline
    {
      "description" : "split message content",
      "processors": [
        {
          "grok": {
              "field": "message",
              "patterns": ["\"agentId:%{NOTSPACE:agentId}\""]
            }
        }
       }
      ]
    }
5

If you use logstash, put the script in to the

    filter {
       grok {
            match => { "message" => "\"agentId:%{NOTSPACE:agentId}\"}" 
       }
    }

Regards,
Fadjar Tandabawana

6

Hi @fadjar340,

Thanks for reply but i configured everything in filebeat so i just wanted to create it as a ingest pipeline and have to use my pipeline Id in filebeat.

So while creating pipeline with the given grok pattern i am getting error. So could you please help me to achieve this in ingest pipeline methods as i have given my pipeline above.

7

There are unnecessary bracket that make an error

     PUT _ingest/pipeline/dissectpipeline
    {
      "description" : "split message content",
      "processors": [
        {
          "grok": {
              "field": "message",
              "patterns": ["\"agentId:%{NOTSPACE:agentId}\""]
            }
        }
       }  <= unnecessary 
      ]
    }
8

Oh God, simple, Wasted lot more time on this. :joy:
Thanks @fadjar340

9

@fadjar340 - Also created another topic related to dissect pipeline added lnk for your reference that is something similar to this.

multiple pattern dissect processor