# Logstash parse json child element, format and insert into elasticsearch **URL:** \ **Category:** Logstash\ **Created:** [August 16, 2022, 11:05pm UTC](https://discuss.elastic.co/t/logstash-parse-json-child-element-format-and-insert-into-elasticsearch/312230 "2022-08-16T23:05:23Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![babuzrb](https://avatars.discourse-cdn.com/v4/letter/b/bcef8e/32.png) [@babuzrb](https://discuss.elastic.co/u/babuzrb)\ **Post date:** [August 16, 2022, 11:05pm UTC](https://discuss.elastic.co/t/logstash-parse-json-child-element-format-and-insert-into-elasticsearch/312230/1 "2022-08-16T23:05:23Z")
I have a json file like this: ```auto "fruits": { "fruit": [ { "id": 1, "label": "test", "tag": "fine", "start": "4", "end": "9" }, { "id": 2, "label": "test1", "tag": "fine1", "start": "2", "end": "4" } ] } } ``` I have 100s of elements inside "fruit" field. I want to: - insert only the elements inside "fruit" field to the elasticsearch each as an individual doc. I want to use their own id as elasticsearch doc id. - calculate numbers in between "start" and "end" fields, then add those numbers as a comma separated string to a new field inside each doc. The docs I want to insert into elasticsearch will be as follows: ```auto { { "_index" : "my_index", "_type" : "_doc", "_id" : "1", "_score" : 1.0, "_source" : { "id" : "1", "label": "test", "tag": "fine", "start": "4", "end": "9", "diffs": "4,5,6,7,8,9" } }, { "_index" : "my_index", "_type" : "_doc", "_id" : "2", "_score" : 1.0, "_source" : { "id" : "2", "label": "test1", "tag": "fine1", "start": "2", "end": "4", "diffs": "2,3,4" } } } ``` Can anyone help me with the logstash configuration file to achieve the desired output? I am using ELK version 7.x Thanks ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [August 17, 2022, 12:00am UTC](https://discuss.elastic.co/t/logstash-parse-json-child-element-format-and-insert-into-elasticsearch/312230/2 "2022-08-17T00:00:02Z")
> [@babuzrb](#): > > `"diffs": "4,5,6,7,8,9"` Since diffs has multiple values that seems to imply that you are aggregating multiple entries from the [fruits][fruit] array that have the same value of id. If so, how do know from which entry the start and end values should be taken? ---
**Author:** ![babuzrb](https://avatars.discourse-cdn.com/v4/letter/b/bcef8e/32.png) [@babuzrb](https://discuss.elastic.co/u/babuzrb)\ **Post date:** [August 17, 2022, 12:02am UTC](https://discuss.elastic.co/t/logstash-parse-json-child-element-format-and-insert-into-elasticsearch/312230/3 "2022-08-17T00:02:24Z")
no aggregation actually. just need to get all the values within this range including "start" and "end" ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [August 17, 2022, 12:03am UTC](https://discuss.elastic.co/t/logstash-parse-json-child-element-format-and-insert-into-elasticsearch/312230/4 "2022-08-17T00:03:40Z")
OK, then I have no idea what you are trying to do. ---
**Author:** ![babuzrb](https://avatars.discourse-cdn.com/v4/letter/b/bcef8e/32.png) [@babuzrb](https://discuss.elastic.co/u/babuzrb)\ **Post date:** [August 17, 2022, 12:06am UTC](https://discuss.elastic.co/t/logstash-parse-json-child-element-format-and-insert-into-elasticsearch/312230/5 "2022-08-17T00:06:17Z")
well, did you get the other parts of the requirement? can you please help to do that? - insert only the elements inside "fruit" field to the elasticsearch each as an individual doc. I want to use their own id as elasticsearch doc id. ---
**Author:** ![babuzrb](https://avatars.discourse-cdn.com/v4/letter/b/bcef8e/32.png) [@babuzrb](https://discuss.elastic.co/u/babuzrb)\ **Post date:** [August 17, 2022, 12:15am UTC](https://discuss.elastic.co/t/logstash-parse-json-child-element-format-and-insert-into-elasticsearch/312230/6 "2022-08-17T00:15:09Z")
> "how do know from which entry the start and end values should be taken?" "start" and "end" values should be taken from each entry (document). like in the example: for id:1, "start" is 4 and "end" is 9. so "diff" will be all the values in between 4 and 9.. like increment by 1 from "start" till it reaches the "end". @Badger ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [August 17, 2022, 12:17am UTC](https://discuss.elastic.co/t/logstash-parse-json-child-element-format-and-insert-into-elasticsearch/312230/7 "2022-08-17T00:17:21Z")
Use a split filter to create a new event for each entry in the array.... ``` split { field => "[fruits][fruit]" } ``` then use mutate+add\_field to move the items in the array entry to the top level using (or possibly use a [ruby](https://discuss.elastic.co/t/how-to-dynamically-move-nested-key-value-to-root-level/180006/2) filter), then use mutate+remove\_field to delete [fruits]. In the output section use a [sprintf reference](https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html#sprintf) ("%{id}") for the value of the document\_id option on the elasticsearch output. @babuzrb To create [diffs] you could use this ``` ruby { code => ' d = "" for i in event.get("start").to_i .. event.get("end").to_i d += "#{i}," end event.set("diffs", d.chop) ' } ``` I am sure there is some much prettier Ruby idiom that would work just as well ---
**Author:** ![babuzrb](https://avatars.discourse-cdn.com/v4/letter/b/bcef8e/32.png) [@babuzrb](https://discuss.elastic.co/u/babuzrb)\ **Post date:** [August 17, 2022, 12:39am UTC](https://discuss.elastic.co/t/logstash-parse-json-child-element-format-and-insert-into-elasticsearch/312230/8 "2022-08-17T00:39:42Z")
> [@Badger](#): > > then use mutate+add\_field to move the items in the array entry to the top level using (or possibly use a [ruby](https://discuss.elastic.co/t/how-to-dynamically-move-nested-key-value-to-root-level/180006/2) filter), then use mutate+remove\_field to delete [fruits]. > > In the output section use a [sprintf reference](https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html#sprintf) ("%{id}") for the value of the document\_id option on the elasticsearch output. @Badger can you please help by writing the configurations that you've suggested? ---
**Author:** ![babuzrb](https://avatars.discourse-cdn.com/v4/letter/b/bcef8e/32.png) [@babuzrb](https://discuss.elastic.co/u/babuzrb)\ **Post date:** [August 17, 2022, 3:04am UTC](https://discuss.elastic.co/t/logstash-parse-json-child-element-format-and-insert-into-elasticsearch/312230/9 "2022-08-17T03:04:50Z")
> [@Badger](#): > > then use mutate+add\_field to move the items in the array entry to the top level using (or possibly use a [ruby](https://discuss.elastic.co/t/how-to-dynamically-move-nested-key-value-to-root-level/180006/2) filter), then use mutate+remove\_field to delete [fruits]. @Badger , I did that following your reference. All "fruit" list elements are now set to events according to a specific key. When I insert them they are inserted into the elasticsearch including the key under "\_source". If I want only the event "value" to store under \_source and ignore the event "key" what should I do? ---
**Author:** ![babuzrb](https://avatars.discourse-cdn.com/v4/letter/b/bcef8e/32.png) [@babuzrb](https://discuss.elastic.co/u/babuzrb)\ **Post date:** [August 17, 2022, 2:46pm UTC](https://discuss.elastic.co/t/logstash-parse-json-child-element-format-and-insert-into-elasticsearch/312230/10 "2022-08-17T14:46:50Z")
@Badger Thank you very much. Following your instructions I could able to complete all requirements finally. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 14, 2022, 2:47pm UTC](https://discuss.elastic.co/t/logstash-parse-json-child-element-format-and-insert-into-elasticsearch/312230/11 "2022-09-14T14:47:37Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.