# How to map nested json array with http output plugin

**URL:** <https://discuss.elastic.co/t/how-to-map-nested-json-array-with-http-output-plugin/215950>\
**Category:** Logstash\
**Created:** [January 21, 2020, 8:49pm UTC](https://discuss.elastic.co/t/how-to-map-nested-json-array-with-http-output-plugin/215950 "2020-01-21T20:49:18Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![rkhapre](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rkhapre/32/48333_2.png) [@rkhapre](https://discuss.elastic.co/u/rkhapre)\
**Post date:** [January 21, 2020, 8:49pm UTC](https://discuss.elastic.co/t/how-to-map-nested-json-array-with-http-output-plugin/215950/1 "2020-01-21T20:49:18Z")

</div>

Hi All

I have incoming events from input as 2 records

> **Project\_Num, Task Name,Task\_num,Task Date**  
> PRJ-001,Task Name1,111,1-Jan-2020  
> PRJ-001,Task Name2,112,2-Jan-2020

How should i map it with my json payload like below in http output  
Basically, i need my data in below format to form a payload

There can be a cases where i can have 3 records from same Project  
or 4 Records from same project as i need to form a array in json output for payload  
So the mapping should be dynamic

Need to group it dynamically to form a one json record

```
{
    "Project":{
        "Project_Num":"PRJ-001",
		
    "Task":[
        {
            "Task Name":"Task Name1",
            "Task_num":111,
            "Task Date":"1-Jan-2020",
        },
        {
           "Task Name":"Task Name2",
            "Task_num":112,
            "Task Date":"2-Jan-2020",
        }
    ]
}

```

Thanks

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [January 21, 2020, 10:02pm UTC](https://discuss.elastic.co/t/how-to-map-nested-json-array-with-http-output-plugin/215950/2 "2020-01-21T22:02:47Z")

</div>

Use an [aggregate](https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html) filter.

---

<div class="post-metadata">

**Author:** ![rkhapre](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rkhapre/32/48333_2.png) [@rkhapre](https://discuss.elastic.co/u/rkhapre)\
**Post date:** [January 22, 2020, 10:37am UTC](https://discuss.elastic.co/t/how-to-map-nested-json-array-with-http-output-plugin/215950/3 "2020-01-22T10:37:48Z")

</div>

Thanks @Badger for your response, but i am looking some clean solution  
I always had issues in aggregate plugin

If you see the case in example 4

When i have data like this

```
  { "country_name": "France", "town_name": "Paris" }
  { "country_name": "France", "town_name": "Marseille" }
  { "country_name": "USA", "town_name": "New-York" }

```

this groups perfectly with two records

```
  { "country_name": "France", "towns": [{"town_name": "Paris"}, {"town_name": "Marseille"}] }
  { "country_name": "USA", "towns": [{"town_name": "New-York"}] }

```

But when i have same data jumbled it gives wrong info, thats were aggregate plugin is not reliable

> ```
> { "country_name": "France", "town_name": "Paris" }
> { "country_name": "USA", "town_name": "New-York" }
> { "country_name": "France", "town_name": "Marseille" }
> 
> ```

If we have data like above, then i get three records.  
In my case also the country\_name is not necessary will come in sequence always.

Any Idea , how we can map arrays to http output.

---

<div class="post-metadata">

**Author:** ![Fabio-sama](https://avatars.discourse-cdn.com/v4/letter/f/b9e5f3/32.png) [@Fabio-sama](https://discuss.elastic.co/u/Fabio-sama)\
**Post date:** [January 22, 2020, 10:53am UTC](https://discuss.elastic.co/t/how-to-map-nested-json-array-with-http-output-plugin/215950/4 "2020-01-22T10:53:15Z")

</div>

Mhmh I don't think you can achieve something like this with Logstash. The aggregate filter is what best fits the case but, as you said, if events are not ordered (as often happens) the result is not granted (which is why I rarely use it).

I think your best shot is to organize an entity-centric index using Elasticsearch [transformations](https://www.elastic.co/guide/en/elasticsearch/reference/current/transforms.html), but this is not the section where to discuss such stuff.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [January 22, 2020, 2:56pm UTC](https://discuss.elastic.co/t/how-to-map-nested-json-array-with-http-output-plugin/215950/5 "2020-01-22T14:56:03Z")

</div>

> [@rkhapre](#):
>
> If we have data like above, then i get three records.

So look at example 3, not example 4.

---

<div class="post-metadata">

**Author:** ![rkhapre](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rkhapre/32/48333_2.png) [@rkhapre](https://discuss.elastic.co/u/rkhapre)\
**Post date:** [January 22, 2020, 5:55pm UTC](https://discuss.elastic.co/t/how-to-map-nested-json-array-with-http-output-plugin/215950/6 "2020-01-22T17:55:21Z")

</div>

okay, let me try this . My input data has an end , it is API response in json format

---

<div class="post-metadata">

**Author:** ![rkhapre](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rkhapre/32/48333_2.png) [@rkhapre](https://discuss.elastic.co/u/rkhapre)\
**Post date:** [January 25, 2020, 8:43pm UTC](https://discuss.elastic.co/t/how-to-map-nested-json-array-with-http-output-plugin/215950/7 "2020-01-25T20:43:52Z")

</div>

great it worked

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 22, 2020, 8:44pm UTC](https://discuss.elastic.co/t/how-to-map-nested-json-array-with-http-output-plugin/215950/8 "2020-02-22T20:44:01Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.