# Getting nested object with json filter

**URL:** <https://discuss.elastic.co/t/getting-nested-object-with-json-filter/215921>\
**Category:** Logstash\
**Created:** [January 21, 2020, 4:44pm UTC](https://discuss.elastic.co/t/getting-nested-object-with-json-filter/215921 "2020-01-21T16:44:57Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![toms130](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/toms130/32/58507_2.png) [@toms130](https://discuss.elastic.co/u/toms130)\
**Post date:** [January 21, 2020, 4:44pm UTC](https://discuss.elastic.co/t/getting-nested-object-with-json-filter/215921/1 "2020-01-21T16:44:57Z")

</div>

Hi all,

I've got problem when trying to get some nested fields in a json object with json filter.

My field look like this

> {  
> "total\_count" : 3,  
> "orders" : [  
> {  
> "marketplace\_order\_id" : "malikilledme3",  
> "order\_lines" : [  
> {  
> "order\_line\_status\_pending" : "PENDING SHIPMENT",  
> "order\_line\_status\_current" : "TO SHIP",  
> "marketplace\_order\_line\_id" : "malikilledme3-1"  
> },  
> {  
> "order\_line\_status\_pending" : "PENDING SHIPMENT",  
> "order\_line\_status\_current" : "TO SHIP",  
> "marketplace\_order\_line\_id" : "malikilledme3-0"  
> }  
> ]  
> },

Original field is [http][response][body][content], I want to get value from [http][response][body][content][orders][marketplace\_order\_id]

I've extracted top level field , but "orders" value is a nested json, so I tried to parse it ith json filter too

```
json {
        source => "[http][response][body][content]"
        target => "[http][response][json]"
        }

json {
        source => "[http][response][json][orders]"
        target => "[http][response][json][orders_json]"
        }

```

But when I do this, I got a java class error exception.

> 2020-01-21T17:23:37,932][INFO][logstash.agent] Pipelines running {:count=\>1, :running\_pipelines=\>[:api\_ws], :non\_running\_pipelines=\>}  
> [2020-01-21T17:23:43,428][WARN][logstash.filters.json] Error parsing json {:source=\>"[http][response][json][orders]", :raw=\>[{"marketplace\_order\_id"=\>"malikilledme3", "order\_lines"=\>[{"marketplace\_order\_line\_id"=\>"malikilledme3-1", "order\_line\_status\_pending"=\>"PENDING SHIPMENT", "order\_line\_status\_current"=\>"TO SHIP"}, {"marketplace\_order\_line\_id"=\>"malikilledme3-0", "order\_line\_status\_pending"=\>"PENDING SHIPMENT", "order\_line\_status\_current"=\>"TO SHIP"}]}, {"marketplace\_order\_id"=\>"malikilledme\_again", "order\_lines"=\>[{"marketplace\_order\_line\_id"=\>"malikilledme\_again-1", "order\_line\_status\_pending"=\>"PENDING SHIPMENT", "order\_line\_status\_current"=\>"TO SHIP"}, {"marketplace\_order\_line\_id"=\>"malikilledme\_again-0", "order\_line\_status\_pending"=\>"PENDING SHIPMENT", "order\_line\_status\_current"=\>"TO SHIP"}]}, {"marketplace\_order\_id"=\>"ws1-3\_test\_validation1", "order\_lines"=\>[{"marketplace\_order\_line\_id"=\>"ws1-3\_test\_validation1-0", "order\_line\_status\_pending"=\>"PENDING SHIPMENT", "order\_line\_status\_current"=\>"SHIPPED"}, {"marketplace\_order\_line\_id"=\>"ws1-3\_test\_validation1-1", "order\_line\_status\_pending"=\>"PENDING REFUND", "order\_line\_status\_current"=\>"SHIPPED"}]}], :exception=\>java.lang.ClassCastException}

Any idea ?  
regards  
thomas

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [January 21, 2020, 8:18pm UTC](https://discuss.elastic.co/t/getting-nested-object-with-json-filter/215921/2 "2020-01-21T20:18:49Z")

</div>

The first json filter will have parsed all the nested JSON, so the second json filter is never going to work. Note that [http][response][json][orders] is an array, so you will need to access [http][response][json][orders][0].

---

<div class="post-metadata">

**Author:** ![toms130](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/toms130/32/58507_2.png) [@toms130](https://discuss.elastic.co/u/toms130)\
**Post date:** [January 22, 2020, 9:18am UTC](https://discuss.elastic.co/t/getting-nested-object-with-json-filter/215921/3 "2020-01-22T09:18:35Z")

</div>

Oh, ok thanks badger !  
I couldn' t find how to deal with arrays in situation I want, how can I do to get all values of fields marketplace\_order\_line\_id for example ?

> Something like [http][response][json][orders][0][order\_lines][all\_entries][marketplace\_order\_line\_id] ?

---

<div class="post-metadata">

**Author:** ![Fabio-sama](https://avatars.discourse-cdn.com/v4/letter/f/b9e5f3/32.png) [@Fabio-sama](https://discuss.elastic.co/u/Fabio-sama)\
**Post date:** [January 22, 2020, 9:26am UTC](https://discuss.elastic.co/t/getting-nested-object-with-json-filter/215921/4 "2020-01-22T09:26:01Z")

</div>

As @Badger said that is an array, that's why you're receiving that error (you're trying to parse and array as a json).

I'm afraid I didn't really understand what you want in output. You wanna get all values of fields `marketplace_order_line_id` and store them where, precisely?  
You'll most likely need a `ruby` filter and we can help you write it, but we first need to know what you're trying to achieve.

How would you like your output doc to be structured? Paste here the desired output and it'll help us help you 😉

---

<div class="post-metadata">

**Author:** ![toms130](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/toms130/32/58507_2.png) [@toms130](https://discuss.elastic.co/u/toms130)\
**Post date:** [January 22, 2020, 10:00am UTC](https://discuss.elastic.co/t/getting-nested-object-with-json-filter/215921/5 "2020-01-22T10:00:43Z")

</div>

hi @Fabio-sama,  
thanks for answering

Let's take this simplified doc

> {  
> "total\_count" : 3,  
> "offset" : 0,  
> "limit" : 200,  
> "orders" : [  
> {  
> "marketplace\_order\_id" : "A"  
> },  
> {  
> "marketplace\_order\_id" : "B"  
> },  
> "marketplace\_order\_id" : "C"  
> }  
> ]  
> }

What I want to extract in ES is something like this

orders.marketplace\_order\_id: ["A", "B", "C"]

or if not possible in the same field,

orders.marketplace\_order\_id.0 : "A"  
orders.marketplace\_order\_id.1 : "B"  
orders.marketplace\_order\_id.2 : "C"

(I was able to get all values with split filter, but split duplicates all document, and I just want to get all values of 1 field in the same document, so it is probably not goog way to do it for me.)

---

<div class="post-metadata">

**Author:** ![Fabio-sama](https://avatars.discourse-cdn.com/v4/letter/f/b9e5f3/32.png) [@Fabio-sama](https://discuss.elastic.co/u/Fabio-sama)\
**Post date:** [January 22, 2020, 10:08am UTC](https://discuss.elastic.co/t/getting-nested-object-with-json-filter/215921/6 "2020-01-22T10:08:40Z")

</div>

And what about `order_lines_status_pending` and `order_lines_status_current`?

I'm sorry but I gotta insist in asking you to post something like:

**Logstash input:**  
`...your FORMATTED input..`

**Desired output**  
`...your FORMATTED output...`

It makes everything easier for you to better understand what you want, for us to better understand what you want and help you achieve it and for future readers to better understand if they have your same problems.

Thanks 😉

---

<div class="post-metadata">

**Author:** ![toms130](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/toms130/32/58507_2.png) [@toms130](https://discuss.elastic.co/u/toms130)\
**Post date:** [January 22, 2020, 10:09am UTC](https://discuss.elastic.co/t/getting-nested-object-with-json-filter/215921/7 "2020-01-22T10:09:15Z")

</div>

Sorry, I've edited previous post to have something more simple.

---

<div class="post-metadata">

**Author:** ![Fabio-sama](https://avatars.discourse-cdn.com/v4/letter/f/b9e5f3/32.png) [@Fabio-sama](https://discuss.elastic.co/u/Fabio-sama)\
**Post date:** [January 22, 2020, 10:34am UTC](https://discuss.elastic.co/t/getting-nested-object-with-json-filter/215921/8 "2020-01-22T10:34:02Z")

</div>

> [@toms130](#):
>
> orders.marketplace\_order\_id: ["A", "B", "C"]

Ok so, admitting the following is your situation:

**Logstash input:**

```
{
  "total_count": 3,
  "offset": 0,
  "limit": 200,
  "orders": [
    {
      "marketplace_order_id": "A"
    },
    {
      "marketplace_order_id": "B"
    },
    {
      "marketplace_order_id": "C"
    }
  ]
}

```

**Desired output:**

```
{
  "total_count": 3,
  "offset": 0,
  "limit": 200,
  "orders": {
    "marketplace_order_id": ["A", "B", "C"]
  }
}

```

This should do what you want:

```
  ruby {
    code => "
      unless (orders = event.get('orders')).nil?
        orders = { 'marketplace_order_id': orders.map{ |o| o['marketplace_order_id'] }.compact }
        event.set('orders', orders)
      end
    "
  }

```

---

<div class="post-metadata">

**Author:** ![toms130](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/toms130/32/58507_2.png) [@toms130](https://discuss.elastic.co/u/toms130)\
**Post date:** [January 22, 2020, 10:47am UTC](https://discuss.elastic.co/t/getting-nested-object-with-json-filter/215921/9 "2020-01-22T10:47:57Z")

</div>

Thanks so much, It works like a charm !

---

<div class="post-metadata">

**Author:** ![Fabio-sama](https://avatars.discourse-cdn.com/v4/letter/f/b9e5f3/32.png) [@Fabio-sama](https://discuss.elastic.co/u/Fabio-sama)\
**Post date:** [January 22, 2020, 10:49am UTC](https://discuss.elastic.co/t/getting-nested-object-with-json-filter/215921/10 "2020-01-22T10:49:16Z")

</div>

No problem!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 19, 2020, 10:49am UTC](https://discuss.elastic.co/t/getting-nested-object-with-json-filter/215921/11 "2020-02-19T10:49:19Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.