# Nested Json to parse **URL:** \ **Category:** Logstash\ **Created:** [September 22, 2017, 1:43pm UTC](https://discuss.elastic.co/t/nested-json-to-parse/101494 "2017-09-22T13:43:19Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![gius78](https://avatars.discourse-cdn.com/v4/letter/g/7ea924/32.png) [@gius78](https://discuss.elastic.co/u/gius78)\ **Post date:** [September 22, 2017, 1:43pm UTC](https://discuss.elastic.co/t/nested-json-to-parse/101494/1 "2017-09-22T13:43:19Z")
Hi all again, I know this topic has been discussed in tha past, but I am a newbie in this world and could not be able to apply to my case the solutions provided in the old discussions. I have to parse a log like this: `> Sep 22 15:08:11 host04 PWM {"sourceAddress":"10.10.10.5","sourceHost":"10.10.10.5","type":"USER","eventCode":"TOKEN_ISSUED","guid":"3792e530-c41a-4d49-8ac0-d516c9","timestamp":"2017-09-22T13:08:11Z","message":"{\"date\":\"2017-09-22T13:08:11Z\",\"name\":\"NEWUSER_EMAIL\",\"data\": **{\" _______ profileID\":\"WRT\",\"CompanyexternalCountry\":\"cn=AGO,ou=countries,ou=mapping,o=xxx\",\"CompanyexternalBrand\":\"cn=Company,ou=brands,ou=mapping,o=xxx\",\"telephoneNumber\":\"\",\"mail\":\"username@nomail.com\",\"CompanyexternalNote\":\"\",\"givenName\":\"john\",\"CompanyexternalCompany\":\"nosocieta\",\"sn\":\"doe\",\"cn\":\"JOHN DOE\",\"Companyexternalvatnumber\":\"34565432\",\"password1\":\"H4sIAAAAAAAAAAFCAL3_UFdDMjU23suete7Z1gFDva07-7WHvegvewZfvVC2DCEsVew4kAIHzQI9-AebILK9uY0un0E1OQlgAEIAAAA=\"}** ,\"dest\":[[truncated]","narrative":"A token has been issued","xdasTaxonomy":"XDAS_AE_SET_CRED_ACCOUNT","xdasOutcome":"XDAS_OUT_PRIV_GRANTED"}` The problem is what is after the word "data": i am not capable of mapping and parsing that message inside the {}... in my logstash.conf I tried to put: ``` filter { if [type] =="syslog" { grok { match =>["message", "%{CISCOTIMESTAMP:timestamp} %{HOSTNAME:Hostname} %{WORD:Application} %{GREEDYDATA:request1}"] } json{ source => "request1" target => "WRTuser" remove_field=>["request1"] } json { source => "WRTuser.message" target => "data" } } } ``` (2 json filters one after another) I was able to parse all the fields outside the inner message, but nothing inside those {}. Anyone have an idea to suggest? thanks a lot ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [September 25, 2017, 5:48am UTC](https://discuss.elastic.co/t/nested-json-to-parse/101494/2 "2017-09-25T05:48:00Z")
You're using the wrong syntax to address nested fields, see [https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html#logstash-config-field-references](https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html#logstash-config-field-references). ---
**Author:** ![gius78](https://avatars.discourse-cdn.com/v4/letter/g/7ea924/32.png) [@gius78](https://discuss.elastic.co/u/gius78)\ **Post date:** [September 25, 2017, 9:31am UTC](https://discuss.elastic.co/t/nested-json-to-parse/101494/3 "2017-09-25T09:31:25Z")
I've changed the second Json parse to: ``` json { source => "[WRTuser][message]" target => "data" } ``` but looks like nothing is changed ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [September 25, 2017, 9:36am UTC](https://discuss.elastic.co/t/nested-json-to-parse/101494/4 "2017-09-25T09:36:35Z")
As far as I can see, the `WRTuser.message` field is not valid JSON as the `data` field have `**` surrounding the value. ---
**Author:** ![gius78](https://avatars.discourse-cdn.com/v4/letter/g/7ea924/32.png) [@gius78](https://discuss.elastic.co/u/gius78)\ **Post date:** [September 25, 2017, 9:43am UTC](https://discuss.elastic.co/t/nested-json-to-parse/101494/5 "2017-09-25T09:43:32Z")
Sorry dunno where those \*\* came from. another log (today's) `Sep 25 11:14:44 hostname PWM {"sourceAddress":"10.10.10.1","sourceHost":"10.10.10.1","type":"USER","eventCode":"TOKEN_ISSUED","guid":"7f38b26c-65c5-4621-816a-b492cb6c5","timestamp":"2017-09-25T09:14:44Z","message":"{\"date\":\"2017-09-25T09:14:44Z\",\"name\":\"NEWUSER_EMAIL\",\"data\":{\" _______ profileID\":\"WRT\",\"CompanyexternalCountry\":\"cn=AND,ou=countries,ou=mapping,o=com\",\"CompanyexternalBrand\":\"cn=Company,ou=brands,ou=mapping,o=com\",\"telephoneNumber\":\"\",\"mail\":\"bncnv@nomail.com\",\"CompanyexternalNote\":\"\",\"givenName\":\"Bianca\",\"CompanyexternalCompany\":\"snowwhite\",\"sn\":\"Neve\",\"cn\":\"BIANCA NEVE\",\"Companyexternalvatnumber\":\"3443322\",\"password1\":\"H4sIAAAAAAAAAAFCAL3_UFdNLkFFUzEyOF9ITUFDMjU23suete7Z1gFDva07-7WHvegvewZfvVC2DCEsVew4kAIHzQI9-AebILK9uY0un0E1OQlgAEIAAAA=\"},\"dest\":[\"b[truncated]","narrative":"A token has been issued","xdasTaxonomy":"XDAS_AE_SET_CRED_ACCOUNT","xdasOutcome":"XDAS_OUT_PRIV_GRANTED"}` ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [September 25, 2017, 9:49am UTC](https://discuss.elastic.co/t/nested-json-to-parse/101494/6 "2017-09-25T09:49:50Z")
It does not look like that is well-formed JSON in the `message` field either. ---
**Author:** ![gius78](https://avatars.discourse-cdn.com/v4/letter/g/7ea924/32.png) [@gius78](https://discuss.elastic.co/u/gius78)\ **Post date:** [September 25, 2017, 10:15am UTC](https://discuss.elastic.co/t/nested-json-to-parse/101494/7 "2017-09-25T10:15:22Z")
with an online parser, looks like it's correctly parsed if pasted starting from the { ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [September 25, 2017, 10:22am UTC](https://discuss.elastic.co/t/nested-json-to-parse/101494/8 "2017-09-25T10:22:13Z")
I parsed the initial JSON, and this gave me the following string for the `message` field, which is not valid JSON: ```auto "{\"date\":\"2017-09-25T09:14:44Z\",\"name\":\"NEWUSER_EMAIL\",\"data\":{\" _______ profileID\":\"WRT\",\"CompanyexternalCountry\":\"cn=AND,ou=countries,ou=mapping,o=com\",\"CompanyexternalBrand\":\"cn=Company,ou=brands,ou=mapping,o=com\",\"telephoneNumber\":\"\",\"mail\":\"bncnv@nomail.com\",\"CompanyexternalNote\":\"\",\"givenName\":\"Bianca\",\"CompanyexternalCompany\":\"snowwhite\",\"sn\":\"Neve\",\"cn\":\"BIANCA NEVE\",\"Companyexternalvatnumber\":\"3443322\",\"password1\":\"H4sIAAAAAAAAAAFCAL3_UFdNLkFFUzEyOF9ITUFDMjU23suete7Z1gFDva07-7WHvegvewZfvVC2DCEsVew4kAIHzQI9-AebILK9uY0un0E1OQlgAEIAAAA=\"},\"dest\":[\"b[truncated]" ``` It starts with a `{`, but does not end with a `}`. ---
**Author:** ![gius78](https://avatars.discourse-cdn.com/v4/letter/g/7ea924/32.png) [@gius78](https://discuss.elastic.co/u/gius78)\ **Post date:** [September 29, 2017, 7:10am UTC](https://discuss.elastic.co/t/nested-json-to-parse/101494/9 "2017-09-29T07:10:36Z")
Ok at the end I addressed the problem. My application had a configuration parameter that truncated the "inner" json message after 900 chars. I changed with 2048 and now the json is correctly formatted. thanks a lot ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 27, 2017, 7:10am UTC](https://discuss.elastic.co/t/nested-json-to-parse/101494/10 "2017-10-27T07:10:49Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.