Logstash filter json warning

Elastic Stack Logstash
1

Using ELK 7.13.2
The logstash is consuming logs from kafka in json format earlier it was parsing the data but now it is not working

[2021-07-22T12:18:44,401][WARN ][logstash.filters.json    ][ise][d51b308934c5e59604230fc981ad93b4109f63ea34e95a917c831d9b539476ad] Error parsing json {:source=>"message", :raw=>["{\"time\":\"2021-07-21T18:01:08+05:30\",\"tags\":\"ise\",\"proxied_srcip\":\"xx.xx.xx.xx\",\"message\":\"xxxxxx,xxxxxxx,xxxxxx,.....,\",\"host\":\"xxx\"}","NOTICE Passed-Authentication: Authentication succeeded"], :exception=>java.lang.ClassCastException}

it is not parsing any data

2

Are you trying to convert a data type to another data type?

3

What is the output you are getting in the message field?

From the error log I think that it is an array

:source=>"message", :raw=>["{\"time\":\"2021-07-21T18:01:08+05:30\",\"tags\":\"ise\",\"proxied_srcip\":\"xx.xx.xx.xx\",\"message\":\"xxxxxx,xxxxxxx,xxxxxx,.....,\",\"host\":\"xxx\"}","NOTICE Passed-Authentication: Authentication succeeded"]

It looks like that the raw part with the content of the field message has two itens.

Can you share your message field?

4

no i am not converting anything.

5

In kafka the message is stored as in json format

{"time":"2021-07-23T12:28:19+05:30","tags":"ise","proxied_srcip":"xx.xx.xx.xx","message":"xxxx","host":"xxxxxx"}

logstash logs

[2021-07-23T14:39:22,724][WARN ][logstash.filters.json    ][ise][d6b83eeca151b16126379e73295e113853498156da958a0e9df1e6c386d3e473] Error parsing json {:source=>"message", :raw=>["{\"time\":\"2021-07-23T14:32:02+05:30\",\"tags\":\"ise\",\"proxied_srcip\":\"xx.xx.xx.xx\",\"message\":\"xxxx\",\"host\":\"xxx\"}", "NOTICE Passed-Authentication: Authentication succeeded"], :exception=>java.lang.ClassCastException: class org.jruby.RubyArray cannot be cast to class org.jruby.RubyIO (org.jruby.RubyArray and org.jruby.RubyIO are in unnamed module of loader 'app')}

I don't know why i am getting this i resolve this by restarting the logstash but after some hours it shows the warning like

[WARN ][logstash.filters.json    ] and [WARN ][logstash.filters.grook    ]
6

Can you share your pipeline?

Also, how are you sending the data to kafka?

7

Hey @leandrojmp the warning is not coming the new issue is filter grok i have to open this issue

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.