# How to parse this json

**URL:** <https://discuss.elastic.co/t/how-to-parse-this-json/194682>\
**Category:** Logstash\
**Created:** [August 10, 2019, 12:50am UTC](https://discuss.elastic.co/t/how-to-parse-this-json/194682 "2019-08-10T00:50:53Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sketchy](https://avatars.discourse-cdn.com/v4/letter/s/258eb7/32.png) [@Sketchy](https://discuss.elastic.co/u/Sketchy)\
**Post date:** [August 10, 2019, 12:50am UTC](https://discuss.elastic.co/t/how-to-parse-this-json/194682/1 "2019-08-10T00:50:53Z")

</div>

I am new to logstash and have been trying to parse the below json with no luck.

```
{
"stat": "user",
"Ref": "USER:[5000,John Smith,3A37332D2659554F9CCE0CD754185269]",
"statistics": [
    "0",
    "",
    "",
    "0",
    0,
    0,
    1,
    "",
    0,
    "",
    "0",
    1,
    "IDLE",
    "5017",
    1505
]

```

}

The main thing I am trying to do is put each statistic into its own field, each line under statistic represents a metric from the source system.

example output:

```
  statistic.Status => "IDLE"
  statistic.Duration => "1505"

```

Can anyone point me in the right direction to get started with this?

---

<div class="post-metadata">

**Author:** ![admlko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/admlko/32/21787_2.png) [@admlko](https://discuss.elastic.co/u/admlko)\
**Post date:** [August 10, 2019, 5:09am UTC](https://discuss.elastic.co/t/how-to-parse-this-json/194682/2 "2019-08-10T05:09:15Z")

</div>

I think you need to convert the statistics array into hash (“key” =\> “value”) and for that you need a second array containing hash keys (column names).

First, you can use json\_lines (or json) codec as input in order to parse the incoming event(s) as json, then in filter section you can use ruby filter to join the two arrays into a hash.  
Something along these lines:

```
Keys = [“Status”, “Duration”, ....]
Event.set( Joined, Hash[keys.zip(event.get(“statistics”).map {|i| i})] )

```

It is probably better to inject the column names (keys array) early on with add\_field etc. At least it is much cleaner and easier to maintain. Then you can just read it in ruby filter (event.get()).

PS. Sorry for the auto-capitalization of the variable names. I’m writing this on my phone.

---

<div class="post-metadata">

**Author:** ![Sketchy](https://avatars.discourse-cdn.com/v4/letter/s/258eb7/32.png) [@Sketchy](https://discuss.elastic.co/u/Sketchy)\
**Post date:** [August 11, 2019, 2:26am UTC](https://discuss.elastic.co/t/how-to-parse-this-json/194682/3 "2019-08-11T02:26:18Z")

</div>

Thanks for the reply but I thought it would be a much easier task to process that data in the statistics field. My initial idea was I thought I would be able to use the CSV filter on the statistics field but that doesn't seem to work. Same with gsub, if I try to do anything to the statistics field I get the error:

```
 gsub mutation is only applicable for strings and arrays of strings, skipping {:field=>"statistics"

```

I'm failing to understand why its so difficult to work with this field, I'm new to logstash but I have been able to get it to do amazing things on logs that are much more complicated than this.

The ruby debug output for the statistics field looks like this, surely there is a simple way I can create a field for each number.

```
"statistics" => [
    [0] "0",
    [1] "",
    [2] "",
    [3] "0",
    [4] 0,
    [5] 0,
    [6] 0,
    [7] "",
    [8] 0,
    [9] "",
    [10] "0",
    [11] 0,
    [12] "IDLE",
    [13] "5017",
    [14] 1051
```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 11, 2019, 12:43pm UTC](https://discuss.elastic.co/t/how-to-parse-this-json/194682/4 "2019-08-11T12:43:38Z")

</div>

```
    mutate { join => { "statistics" => "," } }
    csv { source => "statistics" autogenerate_column_names => true }

```

Obviously you do not actually want autogenerate\_column\_names, you would use the columns option.

---

<div class="post-metadata">

**Author:** ![Sketchy](https://avatars.discourse-cdn.com/v4/letter/s/258eb7/32.png) [@Sketchy](https://discuss.elastic.co/u/Sketchy)\
**Post date:** [August 11, 2019, 2:17pm UTC](https://discuss.elastic.co/t/how-to-parse-this-json/194682/5 "2019-08-11T14:17:56Z")

</div>

That is exactly what I wanted I knew it would be this simple, thanks so much. I must have tried every possible way except this, so simple in the end.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 11, 2019, 2:23pm UTC](https://discuss.elastic.co/t/how-to-parse-this-json/194682/6 "2019-08-11T14:23:17Z")

</div>

My initial reaction was that @admlko was right and this required a ruby filter. It was only when you said you wanted to use a csv filter that I realized that could be done.

---

<div class="post-metadata">

**Author:** ![Sketchy](https://avatars.discourse-cdn.com/v4/letter/s/258eb7/32.png) [@Sketchy](https://discuss.elastic.co/u/Sketchy)\
**Post date:** [August 11, 2019, 2:50pm UTC](https://discuss.elastic.co/t/how-to-parse-this-json/194682/7 "2019-08-11T14:50:24Z")

</div>

Well thanks to the both of you anyway 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 8, 2019, 2:50pm UTC](https://discuss.elastic.co/t/how-to-parse-this-json/194682/8 "2019-09-08T14:50:27Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.