# How to properly parse multi line json?

**URL:** <https://discuss.elastic.co/t/how-to-properly-parse-multi-line-json/182464>\
**Category:** Logstash\
**Created:** [May 23, 2019, 2:14pm UTC](https://discuss.elastic.co/t/how-to-properly-parse-multi-line-json/182464 "2019-05-23T14:14:52Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![ck1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ck1/32/45666_2.png) [@ck1](https://discuss.elastic.co/u/ck1)\
**Post date:** [May 23, 2019, 2:14pm UTC](https://discuss.elastic.co/t/how-to-properly-parse-multi-line-json/182464/1 "2019-05-23T14:14:52Z")

</div>

Hello all,

I am having trouble parsing the following json input in tpc

> ```
> {
> "color": {
> "pixel": "152",
> "shade": "grey"
> },
> "color": {
> "pixel": "157",
> "shade": "dark"
> }
> }
> 
> ```

Result I want : 2 documents into Elastic, 1 document for each color with the field pixel and shade

Things I've attempted

Input tcp json codec : Doesn't work, it treats the event as single line  
Using multiline codec below, it doesn't work either, I am getting json failure

```
codec => multiline {
        pattern => "^\{$"
        negate => true
        what => previous
      } 

```

Using plain input but a split in filter

```
JSON 
filter {
 json {
    source => "message"
 }
 split{
  field=> "colors"
 }
}

```

Any hint, help?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 23, 2019, 3:01pm UTC](https://discuss.elastic.co/t/how-to-properly-parse-multi-line-json/182464/2 "2019-05-23T15:01:09Z")

</div>

> [@ck1](#):
>
> I am having trouble parsing the following json input

A Google search suggests that JSON parsers generally overwrite the value of a duplicate key. That is certainly the case for the parser that logstash uses. I think you will need a custom parser in a ruby filter.

---

<div class="post-metadata">

**Author:** ![ck1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ck1/32/45666_2.png) [@ck1](https://discuss.elastic.co/u/ck1)\
**Post date:** [May 23, 2019, 3:18pm UTC](https://discuss.elastic.co/t/how-to-properly-parse-multi-line-json/182464/3 "2019-05-23T15:18:52Z")

</div>

> [@Badger](#):
>
> a

Yes it overrides the field with duplicated key but using a split should create new events.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 23, 2019, 3:24pm UTC](https://discuss.elastic.co/t/how-to-properly-parse-multi-line-json/182464/4 "2019-05-23T15:24:22Z")

</div>

You cannot split a field that only has a single value.

---

<div class="post-metadata">

**Author:** ![ck1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ck1/32/45666_2.png) [@ck1](https://discuss.elastic.co/u/ck1)\
**Post date:** [May 23, 2019, 3:54pm UTC](https://discuss.elastic.co/t/how-to-properly-parse-multi-line-json/182464/5 "2019-05-23T15:54:29Z")

</div>

I managed to split with with multine pattern =\> '"color"'

Now i am getting this in the message field

"color": {  
"pixel": "152",  
"shade": "grey"  
}

Trying to split it so pixel becomes a key and shade becomes a key

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 23, 2019, 5:45pm UTC](https://discuss.elastic.co/t/how-to-properly-parse-multi-line-json/182464/6 "2019-05-23T17:45:21Z")

</div>

If you transform that into JSON using

```
mutate { gsub => ["message", "\A", "{", "message", "\Z", "}"] }

```

then you can use a json filter to parse it.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 20, 2019, 5:45pm UTC](https://discuss.elastic.co/t/how-to-properly-parse-multi-line-json/182464/7 "2019-06-20T17:45:25Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.