# Flatten json array in Logstash

**URL:** https://discuss.elastic.co/t/flatten-json-array-in-logstash/399
**Category:** Logstash
**Created:** [May 8, 2015, 1:21pm UTC](https://discuss.elastic.co/t/flatten-json-array-in-logstash/399 "2015-05-08T13:21:46Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![rmoff](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rmoff/32/20212_2.png) [@rmoff](https://discuss.elastic.co/u/rmoff)
#### Post date: [May 8, 2015, 1:21pm UTC](https://discuss.elastic.co/t/flatten-json-array-in-logstash/399/1 "2015-05-08T13:21:46Z")

</div>

Is there a way to flatten a json array in Logstash? Use-case is twitter input plugin, wanting to make array data usable in Kibana, c.f. [Show images from URLs in data](https://discuss.elastic.co/t/show-images-from-urls-in-data/337/4?u=rmoff)

thanks.

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [May 8, 2015, 2:09pm UTC](https://discuss.elastic.co/t/flatten-json-array-in-logstash/399/2 "2015-05-08T14:09:40Z")

</div>

I don't understand what your messages look like. In the screenshot in your other thread the only array I see is `indices: [139, 140]`.

---

<div class="post-metadata">

### Author: ![jordansissel](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jordansissel/32/44957_2.png) [@jordansissel](https://discuss.elastic.co/u/jordansissel)
#### Post date: [May 8, 2015, 7:06pm UTC](https://discuss.elastic.co/t/flatten-json-array-in-logstash/399/3 "2015-05-08T19:06:00Z")

</div>

@magnusbaeck In the tweet event, there's an array `{ "entities": { "media": [....] } }` that contains a list of objects (shown in the screenshot)

@rmoff In terms of making this visible in Kibana, I don't know. You could try using the ruby filter to make an array of whatever property you want:

Something like this may help you (I haven't tested it):

```ruby
filter {
  ruby {
    code => 'event["images"] = event["[entities][media]"].collect { |m| m["media_url"] }
  }
}

```

The above should create a field "images" with value as an array of all the `media_url` values.

---

<div class="post-metadata">

### Author: ![rmoff](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rmoff/32/20212_2.png) [@rmoff](https://discuss.elastic.co/u/rmoff)
#### Post date: [May 12, 2015, 8:40am UTC](https://discuss.elastic.co/t/flatten-json-array-in-logstash/399/4 "2015-05-12T08:40:50Z")

</div>

Thanks @jordansissel - good to know at least I'm not missing some obvious functionality 😄  
I tried your snippet (with the missing closing single quote added) with my existing simple config:

```
input {
    twitter {
        consumer_key => "foo"
        consumer_secret => "foo"
        oauth_token => "foo-foo"
        oauth_token_secret => "foo"
        keywords => ["foo","bar"]
        full_tweet => true
    }
}

filter {
          ruby {
            code => 'event["images"] = event["[entities][media]"].collect { |m| m["media_url"] }'
          }
}

output {
        elasticsearch { host => "localhost" protocol => "http" index => "twitter"}
        stdout { codec => rubydebug }
        }

```

but got an error

```
Exception in filterworker {"exception"=>#<NoMethodError: undefined method `collect' for nil:NilClass>,  

```

I'm no Ruby coder so I guess I'll have to leave this one here for now.

thanks again. Robin.

---

<div class="post-metadata">

### Author: ![jordansissel](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jordansissel/32/44957_2.png) [@jordansissel](https://discuss.elastic.co/u/jordansissel)
#### Post date: [May 12, 2015, 5:19pm UTC](https://discuss.elastic.co/t/flatten-json-array-in-logstash/399/5 "2015-05-12T17:19:43Z")

</div>

> [@rmoff](#):
>
> code =\> 'event["images"] = event["[entities][media]"].collect { |m| m["media\_url"] }'

This should work:

```auto
code => 'event["images"] = event["[entities][media]"].collect { |m| m["media_url"] } unless event["[entities][media]"].nil?'

```

---

<div class="post-metadata">

### Author: ![rmoff](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rmoff/32/20212_2.png) [@rmoff](https://discuss.elastic.co/u/rmoff)
#### Post date: [May 12, 2015, 8:49pm UTC](https://discuss.elastic.co/t/flatten-json-array-in-logstash/399/6 "2015-05-12T20:49:29Z")

</div>

Top stuff -- worked a treat. Thanks so much (and for logstash in general).

---

<div class="post-metadata">

### Author: ![rmoff](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rmoff/32/20212_2.png) [@rmoff](https://discuss.elastic.co/u/rmoff)
#### Post date: [July 21, 2016, 9:47am UTC](https://discuss.elastic.co/t/flatten-json-array-in-logstash/399/7 "2016-07-21T09:47:31Z")

</div>

I just came across this post again via a Google search when looking to solve exactly the same issue. On the basis that others might too, I wanted to add to it some ruby code that might be useful for others doing this, that will combine the elements of an array into a single string:

```
ruby {
code => 'event["hashtags_array"] = event["[entities][hashtags]"].collect { |m| m["text"] } unless event["[entities][hashtags]"].nil?
event["hashtags_list"] = event["hashtags_array"].join(",") unless event["[hashtags_array]"].nil?'
}

```

The first line is just as @jordansissel provided me above; the second line is a hack together that I figured out but seems to work. Example input:

```
  "entities": {
    "hashtags": [
      {
        "text": "Iran",
        "indices": [
          18,
          23
        ]
      },
      {
        "text": "News",
        "indices": [
          24,
          29
        ]
      }
    ],

```

Example output:

```
          "hashtags_array" => [
    [0] "Iran",
    [1] "News"
],
            "hashtags_list" => "Iran,News",
```

---

<div class="post-metadata">

### Author: ![berg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/berg/32/14962_2.png) [@berg](https://discuss.elastic.co/u/berg)
#### Post date: [February 8, 2017, 8:52am UTC](https://discuss.elastic.co/t/flatten-json-array-in-logstash/399/8 "2017-02-08T08:52:17Z")

</div>

I am worried this comment may be a little off topic, but thought I'd chime in here because I had a bit of trouble figuring out how to flatten an array in logstash 5. This post was the most relevant that I found that contained the crux of what I was trying to do.

Since the changed Ruby syntax in logstash 5 Event API, I've had to massage the join solution. Of course, I'm just squashing a basic array and don't need to collect the items from a JSON array in the original post. I just needed to check to see if the field is an array, and if so, join the elements together. Kind of fumbling my way through Ruby.

```
ruby => {
  code => 'if(!event.get("[logstash-array][1]").nil?)
    event.set("[logstash-string]", event.get("[logstash-array]").join(","))
    end'
}

```

Maybe it would be worth specifying how to flatten a JSON array with the limited Ruby support in the Event API as well for anyone who needs to do this in version 5. Since the hash syntax is no longer supported, it could be useful for people trying to find this solution moving forward.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 4:28am UTC](https://discuss.elastic.co/t/flatten-json-array-in-logstash/399/9 "2017-07-06T04:28:47Z")

</div>


