Logstash adding unwanted \ \ to field name

Elastic Stack Logstash
1

Hi,

I'm using the http_poller to poll for some data and this is working fine except for one site. The site provides data in JSON and it displays fine in a browser and doesn't show the \ in rawdata view but for some reason logstash is encapsulating every field in \fieldname\ which leans to JSON parse failures.

I tried adding a gsub filter to remove the backslashes but that doesn't work either.

Part of the JSON from stdout with codec => plain. Setting it to json will result in the same but with a parse error. Rest of the config is normal and works fine with other websites.

"[{\"uptime\":55485,\"dataLinkState\":\"Established\",\"state\":\"In Network\",\"netlock\":{\"state\":\"Remote Lock Not Engaged\"}}]"

2

Please show

  • your pipeline configuration,
  • an example document produced by Logstash (use a stdout { codec => rubydebug } output or copy/paste from Kibana's JSON tab), and
  • the JSON parsing error message.
3 
input {
  http_poller {
    urls => {
      url => "myip"
     }
    cacert => "cert.pem"
    truststore => "truststore.jks"
    user => "admin"
    password => "pass"
    truststore_password => "pass"
    schedule => { cron => "50 * * * * UTC"}
    codec => "json"
  }
}



output {
  stdout {
    codec => rubydebug
  }
}

stdoutput redacted

"message" => "[{\"uptime\":58896,\"dataLinkState\":\"Established\",\"state\":\"In Network\",\"netlock\":{\"state\":\"Remote Lock Not Engaged\"}}]"

error
[0] "_jsonparsefailure"

edit: Logstash is adding all the backslashes, they are not in the original data. Backslashes are present with both the json and plain codec.

4

error
[0] "_jsonparsefailure"

No, I want to see the error message from the Logstash log.

5

Ah sorry.

[2018-03-28T16:49:17,326][INFO ][logstash.inputs.http_poller] Registering http_poller Input {:type=>nil, :schedule=>{"cron"=>"50 * * * * UTC"}, :timeout=>nil}
[2018-03-28T16:49:17,333][INFO ][logstash.pipeline        ] Pipeline started {"pipeline.id"=>"main"}
[2018-03-28T16:49:17,369][INFO ][logstash.agent           ] Pipelines running {:count=>1, :pipelines=>["main"]}
[2018-03-28T16:50:01,370][ERROR][logstash.codecs.json     ] JSON parse error, original data now in message field {:error=>#<LogStash::Json::ParserError: java.util.ArrayList cannot be cast to java.util.Map>, :data=>"[{\"uptime\":58896,\"dataLinkState\":\"Established\",\"state\":\"In Network\",\"netlock\":{\"state\":\"Remote Lock Not Engaged\"}}**redacted**
[2018-03-28T16:57:58,411][WARN ][logstash.runner          ] SIGINT received. Shutting down the agent.
[2018-03-28T16:57:58,867][INFO ][logstash.pipeline        ] Pipeline terminated {"pipeline.id"=>"main"}
6

message contains a list rather than an object and that doesn't fly with the json codec (because it can't store an array directly in the message). Use the plain codec and a json filter with target set to a field where you want the array stored.

1 Like
7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.