# Split array with different nested elemetnts **URL:** \ **Category:** Logstash\ **Created:** [February 12, 2022, 11:49pm UTC](https://discuss.elastic.co/t/split-array-with-different-nested-elemetnts/297063 "2022-02-12T23:49:28Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Gil\_Brudner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gil_brudner/32/101668_2.png) [@Gil\_Brudner](https://discuss.elastic.co/u/Gil_Brudner)\ **Post date:** [February 12, 2022, 11:49pm UTC](https://discuss.elastic.co/t/split-array-with-different-nested-elemetnts/297063/1 "2022-02-12T23:49:28Z")
Hi, I have a data stream coming in as array in the following format: ```auto [ { "dataType": 9, "payload": "[{\"isHeadingHome\":0,\"isTesing\":0}]", "data": "9" }, { "dataType": 8, "payload": "[{\"FRONT_LEFT\":0.0,\"FRONT_RIGHT_current\":0.0}]", "data": "8" }, { "dataType": 8, "payload": "[{\"FRONT_LEFT\":0.0,\"FRONT_RIGHT_current\":0.0}]", "data": "8" }, { "dataType": 9, "payload": "[{\"isHeadingHome\":1,\"isTesing\":0}]", "data": "9" } ] ``` I need to split it into multiple events I tried with: split { field =\> "data" } but it does not work and I see the error: ```auto [2022-02-12T23:47:52,591][WARN][logstash.filters.split][main][] Only String and Array types are splittable. field:data is of type = NilClass ``` Any idea what am I missing? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [February 13, 2022, 1:56am UTC](https://discuss.elastic.co/t/split-array-with-different-nested-elemetnts/297063/2 "2022-02-13T01:56:47Z")
If your data is an array then it must be the contents of some named field. That is the field name you need to supply to the split filter. `field:data is of type = NilClass` indicates that a top-level field named [data] does not exist. You may want something like ``` split { field => "yourFieldNameGoesHere" } json { source => "[someField][payload]" target => "anotherField" } mutate { rename => { "[anotherField][0]" => "[anotherField]" } } ``` ---
**Author:** ![Gil\_Brudner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gil_brudner/32/101668_2.png) [@Gil\_Brudner](https://discuss.elastic.co/u/Gil_Brudner)\ **Post date:** [February 13, 2022, 11:41am UTC](https://discuss.elastic.co/t/split-array-with-different-nested-elemetnts/297063/3 "2022-02-13T11:41:20Z")
Thank you for the quick reponse. I have added manipulated the stream to be: ```auto { "data": [ { "dataType": 9, "payload": "[{\"isHeadingHome\":0,\"isTesing\":0}]", "data": "9" }, { "dataType": 8, "payload": "[{\"FRONT_LEFT\":0.0,\"FRONT_RIGHT_current\":0.0}]", "data": "8" }, { "dataType": 8, "payload": "[{\"FRONT_LEFT\":0.0,\"FRONT_RIGHT_current\":0.0}]", "data": "8" }, { "dataType": 9, "payload": "[{\"isHeadingHome\":1,\"isTesing\":0}]", "data": "9" } ] } ``` in the conf file: ```auto split { field => "data" } json { source => "payload" target => "parsedPayload" } ``` Now I am getting: `object mapping for [payload] tried to parse field [payload] as object, but found a concrete value` ---
**Author:** ![Tomo\_M](https://avatars.discourse-cdn.com/v4/letter/t/848f3c/32.png) [@Tomo\_M](https://discuss.elastic.co/u/Tomo_M)\ **Post date:** [February 13, 2022, 12:57pm UTC](https://discuss.elastic.co/t/split-array-with-different-nested-elemetnts/297063/4 "2022-02-13T12:57:44Z")
That error could occur when the output is not compatible with the target Elasticsearch index mappings. You may check the output is what you desired using [stdout output plugin](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-stdout.html). ```auto output { stdout { codec => rubydebug } } ``` If you would share the output and what is the desired output, it will be easier to debug the pipeline. And also it could be caused by the lack of the last mutate filter @Badger shared, resulting that output filter plugin containing original (unparsed) "payload" field. ---
**Author:** ![Gil\_Brudner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gil_brudner/32/101668_2.png) [@Gil\_Brudner](https://discuss.elastic.co/u/Gil_Brudner)\ **Post date:** [February 13, 2022, 9:26pm UTC](https://discuss.elastic.co/t/split-array-with-different-nested-elemetnts/297063/5 "2022-02-13T21:26:51Z")
This the full error: ```auto [2022-02-13T21:20:31,234][WARN][logstash.outputs.elasticsearch][main][ccad6cd3409188a6dd4fe75a73d50d6e92db14a4768c7a636041e969cbd8eb12] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"type9", :routing=>nil}, {"payload"=>"[{\"isHeadingHome\":0,\"isRelocalizing\":0,\"state\":3,\"stateStr\":\"Idle\",\"status\":1,\"statusStr\":\"Docked\"}]", "@version"=>"1", "environment"=>"production", "timestamp"=>1644787048349, "tags"=>["_split_type_failure"], "dataType"=>9, "deviceId"=>"114", "@timestamp"=>2022-02-13T21:17:28.349Z, "zoneId"=>31, "payloadJson"=>[{"statusStr"=>"Docked", "isHeadingHome"=>0, "status"=>1}]}], :response=>{"index"=>{"_index"=>"type9", "_type"=>"_doc", "_id"=>"klX19H4B6euHrZDSaS_E", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"object mapping for [payload] tried to parse field [payload] as object, but found a concrete value"}}}} ``` What I am tryting to achieve is simply to split the array into single documents. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [February 14, 2022, 3:15am UTC](https://discuss.elastic.co/t/split-array-with-different-nested-elemetnts/297063/6 "2022-02-14T03:15:50Z")
See [this](https://discuss.elastic.co/t/getting-illegal-state-exception-error-while-pushing-logs-to-elasticsearch/290029/2) answer for a discussion of mapping exceptions and how to fix them The error is pointing at `{"payload"=>"[{\"isHeadingHome\":0,`. The [payload] field is a string, and Elasticsearch expects it to be an object. Note that the event has a \_split\_type\_failure tag. It may OK that payloadJson is an object -- that's not causing an error. ---
**Author:** ![Tomo\_M](https://avatars.discourse-cdn.com/v4/letter/t/848f3c/32.png) [@Tomo\_M](https://discuss.elastic.co/u/Tomo_M)\ **Post date:** [February 14, 2022, 3:23am UTC](https://discuss.elastic.co/t/split-array-with-different-nested-elemetnts/297063/7 "2022-02-14T03:23:12Z")
That Elasticsearch index error was caused by that the "payload" field of the output was a string. In addition, you have to cope with `_split_type_failure` tag. It occurs when you use split filter on fields other than string or array. the output does not seems to be the result of the shared conf file. There should be some more filters that might cause the error&failure. I wonder if the `_split_type_failure` came from another split filter. When you share the whole pipeline, somebody could check. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 14, 2022, 3:23am UTC](https://discuss.elastic.co/t/split-array-with-different-nested-elemetnts/297063/8 "2022-03-14T03:23:42Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.