Logstash error "expecting List or Map"

Elastic Stack Logstash
1

Hello what could be cause of the error?
Error


LS config

2

It seems that kibana is producing malformed json as log events:
{"type":"response","@timestamp":"2016-04-06T07:20:04+00:00","tags":[],"pid":1745,"method":"get","statusCode":200,"req":{"url":"/","method":"get","headers":{"host":"localhost:5601","user-agent":"Mozilla/5.0 (Windows NT 6.3; WOW64; rv:37.0) Gecko/20100101 Firefox/37.0 open id","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","accept-language":"en-GB,cs;q=0.7,en;q=0.3","accept-encoding":"gzip, deflate, br","dnt":"1","x-forwarded-for":"2001:718:1e01:1725::254","x-forwarded-host":"kibana.farm.particle.cz","x-forwarde d-server":"kibana.farm.particle.cz","connection":"Keep-Alive"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1"},"res":{"statusCode":200,"responseTime":90,"contentLength":9},"message":"GET / 200 90ms - 9.0B"} {"type":"response","@timestamp":"2016-04-06T07:20:04+00:00","tags":[],"pid":1745,"method":"get","statusCode":200,"req":{"url":"/app/kibana","method":"get","headers":{"host":"localhost:5601","user-agent":"Mozilla/5.0 (Windows NT 6.3; WOW64; rv:37.0) Gecko/20100101 Firefox/37.0 openid","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","accept-language":"en-GB,cs;q=0.7,en;q=0.3","accept-encoding":"gzip, deflate, br","dnt":"1","referer":"https://kibana.farm.particle.cz/","x-forwarded-for":"2001:718:1e01:1725::254","x-forwarded-host":"kibana.farm.particle.cz","x-forwarded-server":"kibana.farm.particle.cz","connection":"Keep-Alive"},"remoteAddress":"1{"type":"log","@timestamp":"2016-04-06T09:26:08+00:00","tags":["status","plugin:kibana","info"],"pid":1789,"name":"plugin:kibana","state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}

hint: look for "remoteAddress":"1{"type":
`

3

@mikula are you using v2.3.0? We just released 2.3.1 see https://www.elastic.co/blog/logstash-2.3.1-and-2.2.4-released - if you are running 2.3.0 can you please upgrade to 2.3.1 and see if it solves the problem or you get a similar error?

Colin

4

@mikula also, is it possible for you to systematically reproduce the problem? do you have a sample log that can trigger the problem that you could share with us?

I'd like to understand what exactly is causing that specific error in 2.3.0, but without sample data/logs to reproduce, given the complexity of your config file, it's almost impossible for me to figure the condition which led to that exception.

Thanks,
Colin

5

I have tried this with nightly 2.3.1 and problem was same

6

the sample log entry is in top post (it is line from kibana log)...
when you pus this through json filter it will fail because of "remoteAddress":"1{"type":
(there is no " behind '1')
with error that json elements should be separated by ",".

I think this should be moved to kibana thread because probem is kibana malformed log entry...

PS: log entry once more with problematic part in bold type:

{"type":"response","@timestamp":"2016-04-06T07:20:04+00:00","tags":[],"pid":1745,"method":"get","statusCode":200,"req":{"url":"/app/kibana","method":"get","headers":{"host":"localhost:5601","user-agent":"Mozilla/5.0 (Windows NT 6.3; WOW64; rv:37.0) Gecko/20100101 Firefox/37.0 openid","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8","accept-language":"en-GB,cs;q=0.7,en;q=0.3","accept-encoding":"gzip, deflate, br","dnt":"1","referer":"https://kibana.farm.particle.cz/","x-forwarded-for":"2001:718:1e01:1725::254","x-forwarded-host":"kibana.farm.particle.cz","x-forwarded-server":"kibana.farm.particle.cz","connection":"Keep-Alive"},"remoteAddress":**"1{"**type":"log","@timestamp":"2016-04-06T09:26:08+00:00","tags":["status","plugin:kibana","info"],"pid":1789,"name":"plugin:kibana","state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}

7

for now I just dropped that json filter