# Logstash filtering nested JSON fields (mutate / grok)

**URL:** <https://discuss.elastic.co/t/logstash-filtering-nested-json-fields-mutate-grok/89263>\
**Category:** Logstash\
**Created:** [June 13, 2017, 8:17pm UTC](https://discuss.elastic.co/t/logstash-filtering-nested-json-fields-mutate-grok/89263 "2017-06-13T20:17:33Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![sarveshgeetha](https://avatars.discourse-cdn.com/v4/letter/s/3da27b/32.png) [@sarveshgeetha](https://discuss.elastic.co/u/sarveshgeetha)\
**Post date:** [June 13, 2017, 8:17pm UTC](https://discuss.elastic.co/t/logstash-filtering-nested-json-fields-mutate-grok/89263/1 "2017-06-13T20:17:33Z")

</div>

Hello Folks,

```
input {
  http_poller {
    urls => {
      test2 => {
        method => get
        url => "http://www.coincap.io/history/1day/XRP"
      }
    }
    request_timeout => 60
    schedule => { cron => "* * * * * * UTC"}
    codec => "json"
  }
}
output {
  stdout {
    codec => rubydebug
  }
}

```

I am fetching a JSON data from the above API. The logstash rubydebug output of that JSON is as below:

```
{
             "preminedSig" => false,
                   "isBuy" => true,
                     "dom" => 39,
                  "altCap" => 69121887494,
             "explorerURL" => "https://blockexplorer.com/",
                  "btcCap" => 44395178760,
             "vwapDataBTC" => "2682.6451919194055",
              "capPercent" => "5.70",
                  "supply" => "16389000",
                    "long" => "Bitcoin",
    "cap24hrChangePercent" => "5.70",
                 "twitter" => "",
                    "perc" => "5.70",
              "market_cap" => [
        [0] [
            [0] 1367174841000,
            [1] 1500517590
        ],
        [1] [
            [0] 1367261101000,
            [1] 1575032004
        ],
		],
		    "@timestamp" => 2017-06-13T20:02:21.397Z,
		         "price" => [
		        [0] [
		            [0] 1497286765000,
		            [1] 0.255964
		        ],
		        [1] [
		            [0] 1497287042000,
		            [1] 0.255944
		        ],
		        [2] [
		            [0] 1497287359000,
		            [1] 0.253767
		        ]
}

```

In the above JSON output , i need to parse out the price field which contains two values ([0] , [1] ). One is Timestamp and the other is price. i need to fetch it out those two fields and set the type for one field as TimeStamp and other as number and send to Elastic Search.

I have tried using Mutate and Grok filter. I am unable to attain the result.  
Can anyone of guys help me out ?

> [1] [  
> [0] 1497287042000, ------- Give a seperate field name for this as timestamp  
> [1] 0.255944 -- give a seperate field name for this as price.  
> ],

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 14, 2017, 5:45am UTC](https://discuss.elastic.co/t/logstash-filtering-nested-json-fields-mutate-grok/89263/2 "2017-06-14T05:45:38Z")

</div>

Okay, it's clear what you want to do with a single (timestamp, tuple) entity, but the `price` field contains three such tuples. Do you want to process just one of them or all of them? How?

---

<div class="post-metadata">

**Author:** ![sarveshgeetha](https://avatars.discourse-cdn.com/v4/letter/s/3da27b/32.png) [@sarveshgeetha](https://discuss.elastic.co/u/sarveshgeetha)\
**Post date:** [June 14, 2017, 4:24pm UTC](https://discuss.elastic.co/t/logstash-filtering-nested-json-fields-mutate-grok/89263/3 "2017-06-14T16:24:30Z")

</div>

I need all the data in the price . from price [0] --- price[n] ... each price has two internal fields right. that i want to segregate one as timestamp field and other as a sub\_price fields like that. giving a name to it.

Like this.

```
price [0] = { "timestamp" : 1497286765000(data type to timestamp) , "sub_price":0.2555}.
```

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 15, 2017, 5:47am UTC](https://discuss.elastic.co/t/logstash-filtering-nested-json-fields-mutate-grok/89263/4 "2017-06-15T05:47:12Z")

</div>

Use a ruby filter. Something like this should work:

```nohighlight
event.set('price', event.get('price').collect { |p| {"timestamp" => p[0], "sub_price" => p[1]} })

```

To turn the `timestamp` field into a date field in ES you have a couple of options depending on what you want the documents to look like; you can store either an epoch number and configure the index's mappings to treat it as a date or you can convert the epoch into a human-readable timestamp string and have ES's automapper automatically guess that it's a date.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 13, 2017, 5:47am UTC](https://discuss.elastic.co/t/logstash-filtering-nested-json-fields-mutate-grok/89263/5 "2017-07-13T05:47:36Z")

</div>

Closed on July 13, 2017, 5:47am UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.