# Struggling to parse JSON data with logstash **URL:** \ **Category:** Logstash\ **Created:** [November 8, 2019, 11:01pm UTC](https://discuss.elastic.co/t/struggling-to-parse-json-data-with-logstash/207186 "2019-11-08T23:01:59Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Lee\_Quince](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lee_quince/32/57416_2.png) [@Lee\_Quince](https://discuss.elastic.co/u/Lee_Quince)\ **Post date:** [November 8, 2019, 11:01pm UTC](https://discuss.elastic.co/t/struggling-to-parse-json-data-with-logstash/207186/1 "2019-11-08T23:01:59Z")
So I have a basic http json post that is sent on the format of { "sq": 3, "ht": [ { "si": "90:B6:86:42:2C:9F", "sm": "90B686", "bi": "00:00:00:00:00:00", "ap": 0, "ot": 1473246736, "ct": 1473246736, "sh": -90, "sl": -90, "ss": -90, "cn": 1 }, { "si": "60:A3:7D:D8:A7:F0", "sm": "60A37D", "bi": "28:28:5D:69:14:0A", "ap": 0, "ot": 1473246736, "ct": 1473246736, "sh": -77, "sl": -77, "ss": -77, "cn": 1 }, { "si": "60:D9:C7:9E:DC:E4", "sm": "60D9C7", "bi": "28:28:5D:69:14:0A", "ap": 0, "ot": 1473246736, "ct": 1473246736, "sh": -88, "sl": -88, "ss": -88, "cn": 1 } ], "vs": 3, "zver": 1, "pf": 12, "sn": "84:24:8D:86:45:84" } The ht is the container with the JSON in, what I am trying to do is cycle through each set of data inside "ht" and store as its own document within elastic search, each document should also contain the root level "sn" field and a time stamp. So if i setup logstash to receive the JSON and use a GROK filter with GREEDYDATA it will store the whole thing in a document. for example: { "\_index": "logstash-pbtest-2019.11.08", "\_type": "\_doc", "\_id": "U4QiTW4B9XJ94OOF--q6", "\_version": 1, "\_score": null, "\_source": { "vs": 3, "@version": "1", "ht": [ { "si": "6C:3B:6B:60:E4:31", "ss": -91, "ot": 1573252635, "sl": -91, "sh": -91, "sm": "6C3B6B", "bi": "6C:3B:6B:60:E4:31", "ap": 1, "ct": 1573252635, "cn": 1 }, { "si": "6C:3B:6B:60:E3:E6", "ss": -90, "ot": 1573252635, "sl": -90, "sh": -90, "sm": "6C3B6B", "bi": "6C:3B:6B:60:E3:E6", "ap": 1, "ct": 1573252635, "cn": 1 }, { "si": "AE:86:74:9D:0E:27", "ss": -30, "ot": 1573252635, "sl": -30, "sh": -30, "sm": "AE8674", "bi": "AE:86:74:9D:0E:27", "ap": 1, "ct": 1573252635, "cn": 1 } ], "pf": 12, "headers": { "http\_user\_agent": null, "http\_version": "HTTP/1.1", "http\_accept": "/", "content\_type": "application/json", "request\_path": "/", "request\_method": "POST", "http\_host": "10.129.49.9:8880", "content\_length": "4748" }, "zver": 1, "@timestamp": "2019-11-08T22:29:28.527Z", "sn": "74:67:F7:03:BC:18", "host": "10.2.97.5", "sq": 29 }, "fields": { "@timestamp": [ "2019-11-08T22:29:28.527Z" ] }, "sort": [ 1573252168527 ] } When I come to break this down i use a basic filter / logstash config: input { http { port =\> 8880 codec =\> json } } filter { json { source =\> "[ht]" target =\> "pres" } } output { elasticsearch { hosts =\> ["[elastic01.enc.irondrv.com:9200](http://elastic01.enc.irondrv.com:9200)"] index =\> ["logstash-pbtest-%{+YYYY.MM.dd}"] } stdout { codec =\> rubydebug } } This is where i run into the problem.. I keep getting the following example logged. [2019-11-08T22:42:44,298][WARN][logstash.filters.json][main] Error parsing json {:source=\>"[ht]", :raw=\>[{"sm"=\>"E68D8C", "si"=\>"E6:8D:8C:CB:75:54", "sh"=\>-83, "sl"=\>-83, "ss"=\>-83, "ap"=\>1, "bi"=\>"E6:8D:8C:CB:75:54", "ct"=\>1573253431, "ot"=\>1573253431, "cn"=\>1}, {"sm"=\>"7467F7", "si"=\>"74:67:F7:2A:8E:14", "sh"=\>-49, "sl"=\>-49, "ss"=\>-49, "ap"=\>1, "bi"=\>"74:67:F7:2A:8E:14", "ct"=\>1573253431, "ot"=\>1573253431, "cn"=\>1}, {"sm"=\>"7467F7", "si"=\>"74:67:F7:2A:8E:13", "sh"=\>-52, "sl"=\>-52, "ss"=\>-52, "ap"=\>1, "bi"=\>"74:67:F7:2A:8E:13", "ct"=\>1573253431, "ot"=\>1573253431, "cn"=\>1}, {"sm"=\>"7467F7", "si"=\>"74:67:F7:2A:8E:10", "sh"=\>-30, "sl"=\>-30, "ss"=\>-30, "ap"=\>1, "bi"=\>"74:67:F7:2A:8E:10", "ct"=\>1573253431, "ot"=\>1573253431, "cn"=\>1}, {"sm"=\>"7467F7", "si"=\>"74:67:F7:2A:D7:F0", "sh"=\>-10, "sl"=\>-10, "ss"=\>-10, "ap"=\>1, "bi"=\>"74:67:F7:2A:D7:F0", "ct"=\>1573253431, "ot"=\>1573253431, "cn"=\>1}], :exception=\>java.lang.ClassCastException: class org.jruby.RubyArray cannot be cast to class org.jruby.RubyIO (org.jruby.RubyArray and org.jruby.RubyIO are in unnamed module of loader 'app')} Which I cannot work out why. Obviously its matching the field containing the JSON, but am I making a rooky mistake? Any help would be appreciated. I'm using logstash 7.4.2 on Ubuntu 18.04 installed via APT am I missing a plugin or Ruby app? ---
**Author:** ![Lee\_Quince](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lee_quince/32/57416_2.png) [@Lee\_Quince](https://discuss.elastic.co/u/Lee_Quince)\ **Post date:** [November 8, 2019, 11:22pm UTC](https://discuss.elastic.co/t/struggling-to-parse-json-data-with-logstash/207186/2 "2019-11-08T23:22:14Z")
Ok my bad.. Needed a split filter.. If anybody has a similar issue. This is my initial config that works. input { http { port =\> 8880 codec =\> json } } filter { json { source =\> "message" } split { field =\> "ht" } } output { elasticsearch { hosts =\> ["[elastic01.enc.irondrv.com:9200](http://elastic01.enc.irondrv.com:9200)"] index =\> ["logstash-pbtest-%{+YYYY.MM.dd}"] } stdout { codec =\> rubydebug } } ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [November 9, 2019, 2:15pm UTC](https://discuss.elastic.co/t/struggling-to-parse-json-data-with-logstash/207186/3 "2019-11-09T14:15:50Z")
> [@Lee\_Quince](#): > > Obviously its matching the field containing the JSON No, it is not JSON. The "class org.jruby.RubyArray cannot be cast to class org.jruby.RubyIO" is telling you it is an array. That's because it was already parsed by the json codec. You very rarely need both a json codec and a json filter. Remove the json filter. ---
**Author:** ![Lee\_Quince](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lee_quince/32/57416_2.png) [@Lee\_Quince](https://discuss.elastic.co/u/Lee_Quince)\ **Post date:** [November 9, 2019, 3:19pm UTC](https://discuss.elastic.co/t/struggling-to-parse-json-data-with-logstash/207186/4 "2019-11-09T15:19:57Z")
Cheers for the reply, but the split filter worked for me to resolve, so I assume you are saying the json filter is not actually required? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [November 9, 2019, 3:21pm UTC](https://discuss.elastic.co/t/struggling-to-parse-json-data-with-logstash/207186/5 "2019-11-09T15:21:39Z")
> [@Lee\_Quince](#): > > I assume you are saying the json filter is not actually required? Correct. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 7, 2019, 3:21pm UTC](https://discuss.elastic.co/t/struggling-to-parse-json-data-with-logstash/207186/6 "2019-12-07T15:21:49Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.