# Parse JSON **URL:** \ **Category:** Logstash\ **Created:** [November 8, 2019, 12:02pm UTC](https://discuss.elastic.co/t/parse-json/207104 "2019-11-08T12:02:25Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![Buzz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/buzz/32/57377_2.png) [@Buzz](https://discuss.elastic.co/u/Buzz)\ **Post date:** [November 8, 2019, 12:02pm UTC](https://discuss.elastic.co/t/parse-json/207104/1 "2019-11-08T12:02:25Z")
Hi All, I would like to ask you for a help with parsing data from JSON. I have a log file with huge JSON that contains non parsed parts: dataj.AffectedItems { "Attachments": "ATT00001.png (24733b); ATT00002.jpg (24684b); ATT00003.jpg (15299b); ATT00004.jpg (8273b); ATT00005.jpg (9061b); ATT00006.png (1564b)", "Id": "RgAAAAB8ZguAnTGNQ4JPH2DKODg0BwBtsBPdSBEtT6dXWciKpCD7AAAAqSyqAABQYiqz+jZqRJR42ypOkanPAABJkwQ/AAAJ", "InternetMessageId": "[INX.1556c8dc8d061de33e85714224e.282a.15239.4710.16e@inxserver.com](mailto:INX.1556c8dc8d061de33e85714224e.282a.15239.4710.16e@inxserver.com)", "ParentFolder": { "Id": "LgAAAAB8ZguAnTGNQ4JPH2DKODg0AQBtsBPdSBEtT6dXWciKpCD7AAAAqSyqAAAB", "Path": "\Inbox" }, "Subject": "Subject" } How Can I parse those parts as well? Expected output: **dataj.AffectedItems.Attachemnts:**"ATT00001.png (24733b); ATT00002.jpg (24684b); ATT00003.jpg (15299b); ATT00004.jpg (8273b); ATT00005.jpg (9061b); ATT00006.png (1564b)", **dataj.AffectedItems.Id:**"RgAAAAB8ZguAnTGNQ4JPH2DKODg0BwBtsBPdSBEtT6dXWciKpCD7AAAAqSyqAABQYiqz+jZqRJR42ypOkanPAABJkwQ/AAAJ", **dataj.AffectedItems.InternetMessageId:**"[INX.1556c8dc8d061de33e85714224e.282a.15239.4710.16e31c7b9981ff.bounce\_7960@inxserver.com](mailto:INX.1556c8dc8d061de33e85714224e.282a.15239.4710.16e31c7b9981ff.bounce_7960@inxserver.com)\>" **dataj.AffectedItems.ParentFolder.ID:**"LgAAAAB8ZguAnTGNQ4JPH2DKODg0AQBtsBPdSBEtT6dXWciKpCD7AAAAqSyqAAAB", **dataj.AffectedItems.ParentFolder.Path:**"\Inbox" }, **dataj.AffectedItems.Subject** : "Subject" } Thank you very much Buzz ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [November 8, 2019, 2:59pm UTC](https://discuss.elastic.co/t/parse-json/207104/2 "2019-11-08T14:59:06Z")
You should use a json filter. What problem do you have when you do that? ---
**Author:** ![Buzz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/buzz/32/57377_2.png) [@Buzz](https://discuss.elastic.co/u/Buzz)\ **Post date:** [November 8, 2019, 3:18pm UTC](https://discuss.elastic.co/t/parse-json/207104/3 "2019-11-08T15:18:39Z")
Hi Badger, thank you for answer, i tryed something like this: json { source =\> "datajson" target =\> "json" } json { source =\> "json.AffectedItems" # probably that is problem target =\> "AffectedItems" } But it doesnt work. Buzz ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [November 8, 2019, 3:22pm UTC](https://discuss.elastic.co/t/parse-json/207104/4 "2019-11-08T15:22:40Z")
A json filter will parse all of the nested items within the JSON. There is no need for multiple filters. ---
**Author:** ![Buzz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/buzz/32/57377_2.png) [@Buzz](https://discuss.elastic.co/u/Buzz)\ **Post date:** [November 8, 2019, 4:04pm UTC](https://discuss.elastic.co/t/parse-json/207104/5 "2019-11-08T16:04:11Z")
> [@Badger](#): > > A json filter will parse all of t No it doesnt work in my case, example I showed is after JSON filter. It parsed a lot of items byt that one not. Buzz ---
**Author:** ![Buzz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/buzz/32/57377_2.png) [@Buzz](https://discuss.elastic.co/u/Buzz)\ **Post date:** [November 8, 2019, 4:16pm UTC](https://discuss.elastic.co/t/parse-json/207104/6 "2019-11-08T16:16:25Z")
In log I can see: [2019-11-08T16:19:16,849][WARN][logstash.filters.json] Error parsing json {:source=\>"AffectedItems", :raw=\>[{"Attachments"=\>"ATT00001.png (24733b); ATT00002.jpg (24684b); ATT00003.jpg (15299b); ATT00004.jpg (8273b); ATT00005.jpg (9061b); ATT00006.png (1564b)", "Subject"=\>"Subject", "InternetMessageId"=\>"[INX.1556c8dc8d061de33e85714224e.282a.15239.4710.16e@inxserver.com](mailto:INX.1556c8dc8d061de33e85714224e.282a.15239.4710.16e@inxserver.com)", "ParentFolder"=\>{"Path"=\>"\Inbox", "Id"=\>"LgAAAAB8ZguAnTGNQ4JPH2DKODg0AQBtsBPdSBEtT6dXWciKpCD7AAAAqSyqAAAB"}, "Id"=\>"RgAAAAB8ZguAnTGNQ4JPH2DKODg0BwBtsBPdSBEtT6dXWciKpCD7AAAAqSyqAABQYiqz+jZqRJR42ypOkanPAABJkwQ/AAAJ"}], :exception=\>java.lang.ClassCastException: org.jruby.RubyArray cannot be cast to org.jruby.RubyIO} May be there is a lot of ID fields? Buzz ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [November 8, 2019, 4:24pm UTC](https://discuss.elastic.co/t/parse-json/207104/7 "2019-11-08T16:24:23Z")
> [@Buzz](#): > > :raw=\>[{"Attachments"=\>"ATT00001.png (24733b); ... That's not valid JSON. It is using =\> to separate elements rather than : What does the entire event look like before you start trying to parse it? Please use [markdown](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet) to prevent the browser consuming parts of your message. ---
**Author:** ![Buzz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/buzz/32/57377_2.png) [@Buzz](https://discuss.elastic.co/u/Buzz)\ **Post date:** [November 8, 2019, 4:38pm UTC](https://discuss.elastic.co/t/parse-json/207104/8 "2019-11-08T16:38:54Z")
No it was error from logstash. full JSON of my event is here: `{"GeneralTypeName":"Audit.Exchange","ClientIPAddress":"x.x.x.x","ClientInfoString":"Client=ActiveSync","ClientProcessName":null,"ClientVersion":null,"ExternalAccess":false,"InternalLogonType":0,"LogonType":0,"LogonUserSid":"S-1-5-21-XXXXXXXXXXXX-XXXXXXXX-XXXXXXXXX-XXXXXXX","MailboxGuid":"1111111111111111111111111","MailboxOwnerSid":"S-1-5-21-XXXXXXXXXXXX-XXXXXXXX-XXXXXXXXX-XXXXXXX","MailboxOwnerUPN":"upn@company.com","OrganizationName":"company.onmicrosoft.com","OriginatingServer":"VI1PR0502MB3950 (15.20.2387.009)\r\n","SessionId":null,"AffectedItems":[{"Id":"adf213as2d1f321asd32f132as1df23as1df321asd32f1","Attachments":"ATT00001.png (24733b); ATT00002.jpg (24684b); ATT00003.jpg (15299b); ATT00004.jpg (8273b); ATT00005.jpg (9061b); ATT00006.png (1564b)","InternetMessageId":"","ParentFolder":{"Id":"1231231321323121321321231321321321sdfsdfsdf","Path":"\\Inbox"},"Subject":"Subject"}],"CrossMailboxOperation":false,"Folder":{"Id":"adsf213asdf123asdf132as1df32a1sdf321","Path":"\\Inbox"},"CreationTime":"2019-11-03T18:04:27","Id":"asdasdasd23123a1sd32132a1sd321","Operation":"MoveToDeletedItems","OrganizationId":"1231231231321321321231321321321","RecordType":3,"UserKey":"100300009307E306","UserType":0,"Version":"1","Workload":"Exchange","ResultStatus":"Succeeded","ObjectId":null,"UserId":"upn@company.com","ClientIP":"x.x.x.x","Scope":0}` JSON filter parse almost all except AffectedItems: { "Attachments": "ATT00001.png (24733b); ATT00002.jpg (24684b); ATT00003.jpg (15299b); ATT00004.jpg (8273b); ATT00005.jpg (9061b); ATT00006.png (1564b)", "Subject": "Subject", "InternetMessageId": "[INX.123121312313213213@inxserver.com](mailto:INX.123121312313213213@inxserver.com)", "ParentFolder": { "Path": "\Inbox", "Id": "1231231321323121321321231321321321sdfsdfsdf" }, "Id": "1231231321323121321321231321321321sdfsdfsdf" } Buzz ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [November 8, 2019, 4:51pm UTC](https://discuss.elastic.co/t/parse-json/207104/9 "2019-11-08T16:51:04Z")
But a json filter does parse that. ``` filter { json { source => "message" remove_field => ["message"] } } ``` produces this rubydebug output ``` { "LogonType" => 0, "MailboxOwnerUPN" => "upn@company.com", "AffectedItems" => [ [0] { "Subject" => "Subject", "InternetMessageId" => "", "Attachments" => "ATT00001.png (24733b); ATT00002.jpg (24684b); ATT00003.jpg (15299b); ATT00004.jpg (8273b); ATT00005.jpg (9061b); ATT00006.png (1564b)", "ParentFolder" => { "Path" => "\\Inbox", "Id" => "1231231321323121321321231321321321sdfsdfsdf" }, "Id" => "adf213as2d1f321asd32f132as1df23as1df321asd32f1" } ], "ExternalAccess" => false, "RecordType" => 3, ``` ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [November 9, 2019, 2:13pm UTC](https://discuss.elastic.co/t/parse-json/207104/10 "2019-11-09T14:13:58Z")
> [@Buzz](#): > > [logstash.filters.json] Error parsing json {:source=\>"AffectedItems", :raw=\>[{"Attachments"=\>"ATT00001.png (24733b); ..."}], :exception=\>java.lang.ClassCastException: org.jruby.RubyArray cannot be cast to org.jruby.RubyIO} That error is telling that the thing you are trying to parse is not JSON. It is not JSON because it has already been parsed. For example, if I run this configuration ``` input { generator { count => 1 lines => ['{"AnArray":[{"a":1}, {"b":2}]}' ] } } filter { json { source => "message" } } output { stdout { codec => rubydebug { metadata => false } } } ``` I get ``` "AnArray" => [ [0] { "a" => 1 }, [1] { "b" => 2 } ], ``` in my rubydebug output. If I then add a second json filter ``` json { source => "AnArray" } ``` I get the error Error parsing json {:source=\>"AnArray", :raw=\>[{"a"=\>1}, {"b"=\>2}], :exception=\>java.lang.ClassCastException: org.jruby.RubyArray cannot be cast to org.jruby.RubyIO} ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 7, 2019, 2:14pm UTC](https://discuss.elastic.co/t/parse-json/207104/11 "2019-12-07T14:14:02Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.