Logstash Grok Pattern Not Working(Regex)

[Prakash111]

Hi Team, I have few logs which is having same pattern
DATE LOGLEVEL textData JSONDATA

i want to write grok pattern, it should pass all these 3 logs

my grok pattern

%{TIMESTAMP_ISO8601:time}\s*\[%{LOGLEVEL:logleve}\]\s*%{DATA:textData}\s*%{GREEDYDATA:jsonMessage}

note: this GREEDYDATA ie. jsonMessage, i will take care of that but i need help in DATA ie. textData part.
in the context of textData, I have few spaces and special chars and numbers in text data

2022-12-29T06:27:38.500Z	[INFO]	NewClient	{"Pod": "659bd6cf85-7hhf9", "Service": "service1", "service.client.connected": 4}

2022-12-30T07:25:52.349Z	[INFO]	Get Compatible doc for event querySelect count(flag) from service1 where id = 'ABC-43862' and flag = 'true'	{"Pod": "7f988d47b-psmvm", "Service": "service1"}

2022-12-30T07:25:52.349Z	[INFO]	Before checking in cache	{"Pod": "7f988d47b-psmvm", "Service": "service1", "doc ID: ": "ACD-43862", "MyFlag": false}

I need working grok pattern, to pass all 3 logs, Please help me to solve this issue.

[Rios]

Your lines are tab separated values,check here. You can use:

1. grok:

 grok { 
      match => { "message" => "%{TIMESTAMP_ISO8601:time}\t+\[%{LOGLEVEL:loglevel}\]\t+%{DATA:textData}\t+%{GREEDYDATA:jsonMessage}" }

    }

2. dissect

 dissect {
	mapping => {
        "message" => "%{time}	[%{loglevel}]	%{textData}	%{jsonMessage}"
	}
 }

Also csv plugin

[Prakash111]

Hi @Rios , Thanks for your reply.
Let me explain my issue clearly,
in 1st log line, "NewClient" is there
in 2nd log line, "Get Compatible doc for event querySelect count(flag) from service1 where id = 'ABC-43862' and flag = 'true'" is there
in 3nd log line "Before checking in cache" is there

so here if I use %{DATA:textData} or %{WORD:textData} it will consider only one word, but not all words, some times whole string may have spaces and few special chars too.

so it has to detect all words in that string .
it should not disturb beside JSON data.

i hopp you understand .

[Rios]

This is the result:

{
           "time" => "2022-12-30T07:25:52.349Z",
     "@timestamp" => 2022-12-30T10:18:49.204987800Z,
    "jsonMessage" => "{\"Pod\": \"7f988d47b-psmvm\", \"Service\": \"service1\"}\r",
       "loglevel" => "INFO",
       "textData" => "Get Compatible doc for event querySelect count(flag) from service1 where id = 'ABC-43862' and flag = 'true'"
}
{
           "time" => "2022-12-29T06:27:38.500Z",
     "@timestamp" => 2022-12-30T10:18:49.204987800Z,
    "jsonMessage" => "{\"Pod\": \"659bd6cf85-7hhf9\", \"Service\": \"service1\", \"service.client.connected\": 4}\r",
       "loglevel" => "INFO",
       "textData" => "NewClient"
}
{
           "time" => "2022-12-30T07:25:52.349Z",
     "@timestamp" => 2022-12-30T10:18:49.204987800Z,
    "jsonMessage" => "{\"Pod\": \"7f988d47b-psmvm\", \"Service\": \"service1\", \"doc ID: \": \"ACD-43862\", \"MyFlag\": false}\r",
       "loglevel" => "INFO",
       "textData" => "Before checking in cache"
}

[Prakash111]

Thanks @Rios