How to remove extra characters by using grok pattern in logstash

Elastic Stack Logstash
1

Hi
I trying to write grok pattern to remove some extra characters in between the message by using grok pattern. after removing these extra characters i want to apply json filter. I am sharing sample logs below.

2024-08-20T08:55:03.642963576Z stdout F 08:55:03.642 [xxxx-qwe-9090-exec-8] INFO com.xxxx.vnb.abconvert.ABCVDGontroller - {"logtype":"abc","component":"jkl","request":{"method":"POST","path":"/ops","headers":{"test":"artyucff"}},"response":{"status":400},"runtimes":{"totalTime":12}}

I want to remove below portion from above message
08:55:03.642 [http-qwe-9090-exec-8] INFO com.xxxx.vnb.abconvert.ABCVDGontroller -

I am sharing created grok pattern below.
%{TIMESTAMP_ISO8601:timestamp}%{SPACE}%{WORD:[@metadata][logoutputtype]}%{SPACE}%{WORD:[@metadata][loglevel]}%{SPACE}%{DATA:[@metadata]%{SPACE}-%{SPACE}} %{GREEDYDATA:jsonmessage}"

Could you please guide me to crate grok pattern for above message?

2

You can remove extra info

POST /_ingest/pipeline/_simulate
{
  "pipeline": {
    "description": "Parse log message to JSON",
    "processors": [
      {
        "grok": {
          "field": "message",
          "patterns": [
            """%{TIMESTAMP_ISO8601:timestamp}Z stdout %{WORD:log_level} %{TIME:time} \\[%{DATA:thread_name}\\] %{LOGLEVEL:level} %{JAVACLASS:logger} - %{GREEDYDATA:json_message}"""
          ]
        }
      },
      {
        "json": {
          "field": "json_message",
          "target_field": "json_data"
        }
      }
    ]
  },
  "docs": [
    {
      "_source": {
        "message": """2024-08-20T08:55:03.642963576Z stdout F 08:55:03.642 [xxxx-qwe-9090-exec-8] INFO com.xxxx.vnb.abconvert.ABCVDGontroller - {"logtype":"abc","component":"jkl","request":{"method":"POST","path":"/ops","headers":{"test":"artyucff"}},"response":{"status":400},"runtimes":{"totalTime":12}}"""
      }
    }
  ]
}
3

Hi,

try with this:

%{TIMESTAMP_ISO8601:timestamp} %{WORD:[@metadata][logoutputtype]} F %{DATA:[@metadata][discard]} - %{GREEDYDATA:jsonmessage}

image
image1416×285 18.1 KB

Regards

4

Hi yago,
Thanks for replying. It is working fine :slightly_smiling_face:

Thanks
upendar