Log contains normal text and JSON

[jof300]

Hello,

We have some application logs which contains one part in normal text and sometimes (when ERROR) one part in JSON format. Here is an exemple :

2018-03-15 17:48:21.087 ERROR com.example.heraclesservice.account.AccountServiceImpl#forcePush : [] status 400 reading ContactFeignClient#create(Contact); content:
[{"message":"blablalbla.","errorCode":"DUPLICATES_DETECTED","fields":[]}]

I created a logstash grok filter :

grok {
patterns_dir => "./patterns/"
match => { "message" => "%{HERACLES}" }
}

and the pattern I have created :

ARGUMENTS \[.*\]
HERACLES %{TIMESTAMP_ISO8601:timestamp} \s?%{LOGLEVEL:loglevel} %{JAVACLASS:javaclass}#%{JAVAMETHOD:javamethod} : %{ARGUMENTS:args} %{GREEDYDATA:transaction_msg}

Everything is parsed correctly except the JSON part. I see this in Kibana :

transaction_msg status 400 reading ContactFeignClient#create(Contact); content:

The JSON part is not taken, it's truncated..

I thought GREEDYDATA would take it.... Any idea how to resolve this ?

Thanks.

Regards

[paz]

Is your log message multiline?
Your example message points to that, and the GREEDYDATA part grabs everything up to where the newline character seems to be if that's the case,

[jof300]

Hello,

No, the message is in one line.

Regards

[magnusbaeck (Magnus Bäck)]

No, the message is in one line.

The example you posted sure has a newline character in it and it would explain the behavior you're seeing perfectly.

[jof300]

Yes, it seems you are right.

When I open the log in notepad, the log is one line. On a Linux, in two lines as you said.

I'll check why .

Thank you.