# How to parse the multiline and nested json file

**URL:** https://discuss.elastic.co/t/how-to-parse-the-multiline-and-nested-json-file/297542
**Category:** Logstash
**Created:** [February 17, 2022, 8:00pm UTC](https://discuss.elastic.co/t/how-to-parse-the-multiline-and-nested-json-file/297542 "2022-02-17T20:00:34Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![ycui56](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ycui56/32/101965_2.png) [@ycui56](https://discuss.elastic.co/u/ycui56)
#### Post date: [February 17, 2022, 8:00pm UTC](https://discuss.elastic.co/t/how-to-parse-the-multiline-and-nested-json-file/297542/1 "2022-02-17T20:00:34Z")

</div>

Hi,

I read lots of posts on the similar topics, but I still have problems to figure it out. So I have to post this one and hopefully get help from here. Thanks a lot in advance.

_ **Jason file:** _  
{  
"lable1":  
{  
d1: 0,  
d2: 0,  
d3: 0,  
d4: 0  
},  
"lable2":  
{  
d2: 0,  
d4: 0  
},  
"lable3":  
{  
d1: 0,  
d3: 0  
}  
}

_ **Expect Output in Kibana:** _  
3 separate Documents:  
For example:  
dname: label1  
d1: 0  
d2: 0  
d3: 0  
d4: 0

My current logstash.conf:

```auto
input {
        tcp {
                port => 9400
        }

        file {
            type => "json"
            path => "./temp/*.json"
            codec => multiline {
            pattern => "^\{|\}"
            negate => true
            what => "previous"
            auto_flush_interval => 1
            #multiline_tag => ""
            }
            start_position => "beginning"
            sincedb_path => "/dev/null"
        }
}

filter {
      #mutate { gsub => ["message", "\A", "{", "message", "\Z", "}"] }
      json { source => "message"}
}

```

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [February 17, 2022, 8:22pm UTC](https://discuss.elastic.co/t/how-to-parse-the-multiline-and-nested-json-file/297542/2 "2022-02-17T20:22:31Z")

</div>

Instead of trying to use the multiline filter to pick out each object and then mutate it into valid JSON, I would take the whole file as a single event and then restructure the result. For the input I would use

```
    file {
        path => "/home/ec2-user/t.test/foo.txt"
        codec => multiline {
            pattern => "^Spalanzani"
            negate => true
            what => "previous"
            auto_flush_interval => 1
            multiline_tag => ""
        }
        start_position => "beginning"
        sincedb_path => "/dev/null"
    }

```

Note that file paths must be absolute. You cannot use "./temp/\*.json"

Then restructure it using

```
    json { source => "message" target => "[@metadata][json]" remove_field => ["message"] }
    ruby {
        code => '
            json = event.remove("[@metadata][json]")
            if json.is_a? Hash
                newJson = []
                json.each { |k, v|
                    newJson << v.merge({ "dname" => k })
                }
                event.set("[@metadata][dname]", newJson)
            end
        '
    }
    split { field => "[@metadata][dname]" }
    ruby {
        code => '
            d = event.remove("[@metadata][dname]")
            if d.is_a? Hash
                d.each { |k, v|
                    event.set(k, v)
                }
            end
        '
    }
```

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 17, 2022, 8:23pm UTC](https://discuss.elastic.co/t/how-to-parse-the-multiline-and-nested-json-file/297542/3 "2022-03-17T20:23:25Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
