# Filebeat to logstash problem to parse json message **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [December 13, 2017, 8:37pm UTC](https://discuss.elastic.co/t/filebeat-to-logstash-problem-to-parse-json-message/111642 "2017-12-13T20:37:20Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Fabio\_Scoppetta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fabio_scoppetta/32/25528_2.png) [@Fabio\_Scoppetta](https://discuss.elastic.co/u/Fabio_Scoppetta)\ **Post date:** [December 13, 2017, 8:37pm UTC](https://discuss.elastic.co/t/filebeat-to-logstash-problem-to-parse-json-message/111642/1 "2017-12-13T20:37:20Z")
Hello I am trying to parse json logs in filebeat and send to elastic without logstash If a have message like in below works fine: {"@timestamp":"2017-12-11T19:52:51.262-02:00","@version":1,"message":"foo","logger\_name":"xpto","thread\_name":"thread-1","level":"INFO","level\_value":20000,"app\_name":"foo-api","app\_version":"2.4.5"} But if a have json inside a message tag the message not parse {"@timestamp":"2017-12-12T17:29:24.949-02:00","@version":1,"**message":"{"signatu re": "foo", "args": "[foo [ bar: xxx ... "....** Have some way to parse json inside a message tag? I am ussing filebeat 5.6.3 and configuration like below paths: - /var/log/foo/bar.log document\_type: json json.keys\_under\_root: true json.add\_error\_key: true json.message\_key: "message" And I try to use decode\_json\_fields also but didint work ---
**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\ **Post date:** [December 13, 2017, 8:44pm UTC](https://discuss.elastic.co/t/filebeat-to-logstash-problem-to-parse-json-message/111642/2 "2017-12-13T20:44:59Z")
> [@Fabio\_Scoppetta](#): > > And I try to use decode\_json\_fields also but didint work This is kind of use case that decode\_json\_fields is for so I would expect it to work if the embedded JSON is valid and properly escaped. Can you provide a log sample that isn't truncated so that someone can try to replicate the issue? Were there any errors/warnings in the Filebeat log? ---
**Author:** ![Fabio\_Scoppetta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fabio_scoppetta/32/25528_2.png) [@Fabio\_Scoppetta](https://discuss.elastic.co/u/Fabio_Scoppetta)\ **Post date:** [December 13, 2017, 9:37pm UTC](https://discuss.elastic.co/t/filebeat-to-logstash-problem-to-parse-json-message/111642/3 "2017-12-13T21:37:15Z")
Hello Andrew, thanks for answer I found in log debug 2017/12/13 21:23:13.770045 processor.go:67: DBG fail to apply processor decode\_json\_fields=message: invalid character '\n' in string literal 2017/12/13 21:23:13.771363 client.go:214: DBG Publish: { Its possible to escape or remove this characters ? ---
**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\ **Post date:** [December 13, 2017, 9:43pm UTC](https://discuss.elastic.co/t/filebeat-to-logstash-problem-to-parse-json-message/111642/4 "2017-12-13T21:43:02Z")
There's no way to do this in Filebeat. You would need Logstash. But shouldn't it be the responsibility of the thing creating these logs to write valid JSON? ---
**Author:** ![Fabio\_Scoppetta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fabio_scoppetta/32/25528_2.png) [@Fabio\_Scoppetta](https://discuss.elastic.co/u/Fabio_Scoppetta)\ **Post date:** [December 13, 2017, 9:52pm UTC](https://discuss.elastic.co/t/filebeat-to-logstash-problem-to-parse-json-message/111642/5 "2017-12-13T21:52:01Z")
Ok This is a example of log I remove \n in this example {"@timestamp":"2017-12-12T17:29:24.949-02:00","@version":1,"message":"{"signature": "aaa", "args": "123 | abc"}","logger\_name":"br.com.ab.abcd.log.LoggingAspect","thread\_name":"aaa-7-thread-1","level":"WARN","level\_value":30000,"app\_name":"ab-abcd-api","app\_version":"2.4.5"} but also not work ---
**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\ **Post date:** [December 13, 2017, 9:55pm UTC](https://discuss.elastic.co/t/filebeat-to-logstash-problem-to-parse-json-message/111642/6 "2017-12-13T21:55:55Z")
The contents of `message` are not properly escaped. The inner quotes should be escaped. See the [linter](https://jsonlint.com/?json=%7B%22@timestamp%22:%222017-12-12T17:29:24.949-02:00%22,%22@version%22:1,%22message%22:%22%7B%22signature%22:%20%22aaa%22,%20%22args%22:%20%22123%20%7C%20abc%22%7D%22,%22logger_name%22:%22br.com.ab.abcd.log.LoggingAspect%22,%22thread_name%22:%22aaa-7-thread-1%22,%22level%22:%22WARN%22,%22level_value%22:30000,%22app_name%22:%22ab-abcd-api%22,%22app_version%22:%222.4.5%22%7D) output. ---
**Author:** ![Fabio\_Scoppetta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fabio_scoppetta/32/25528_2.png) [@Fabio\_Scoppetta](https://discuss.elastic.co/u/Fabio_Scoppetta)\ **Post date:** [December 13, 2017, 9:58pm UTC](https://discuss.elastic.co/t/filebeat-to-logstash-problem-to-parse-json-message/111642/7 "2017-12-13T21:58:51Z")
It works! Thanks for pacience and help ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 10, 2018, 9:59pm UTC](https://discuss.elastic.co/t/filebeat-to-logstash-problem-to-parse-json-message/111642/8 "2018-01-10T21:59:05Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.