# Need to Parse a nested JSON message in #Logstash

**URL:** <https://discuss.elastic.co/t/need-to-parse-a-nested-json-message-in-logstash/312979>\
**Category:** Logstash\
**Created:** [August 26, 2022, 2:52am UTC](https://discuss.elastic.co/t/need-to-parse-a-nested-json-message-in-logstash/312979 "2022-08-26T02:52:34Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![pavanKumar2K](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pavankumar2k/32/110025_2.png) [@pavanKumar2K](https://discuss.elastic.co/u/pavanKumar2K)\
**Post date:** [August 26, 2022, 2:52am UTC](https://discuss.elastic.co/t/need-to-parse-a-nested-json-message-in-logstash/312979/1 "2022-08-26T02:52:34Z")

</div>

Hello ,

I am trying to send my logs files ( .txt / json files ) to Logstash via Filebeat

my sample log structure is as below :

```auto
{"LogDetails":{"transaction-id":"1234","channel-id":"abc","APIName":"testapi","OperationName":"get","Timestamp":"2021-02-22 10:07:42.949352","BackendName":"NA","LogType":"Request","Status":"0","Parameters":{"id":"121"}}}

```

flow - file beats \>\> log stash \>\> Elasticsearch

In Elasticsearch i need the index fields to contain the fields of my logs and the log file should not be as a single message inside "message" field in Elasticsearch

here is my logstash.conf file \>

```auto
input {
  beats {
    port => 5044
  }
}
filter {
    grok {
        match => { "message" => "%{GREEDYDATA:LogData}"}
    }
    json {
        source => "LogData"
        target => "LogData"
        skip_on_invalid_json => true
    }
}
output {

    elasticsearch {
        ilm_enabled => true
        index => "gorktest1"
        hosts => ["http://localhost:9200"]
    }
    stdout {}
}

```

in filebeat.yaml \>

```auto
filebeat.inputs:
- type: filestream
  enabled: true
  paths:
    - C:\Users\nm\documents\{folder}\*
  json.keys_under_root: true

output.logstash:
  hosts: ["localhost:5044"]

processors:
  #- add_host_metadata:
     # when.not.contains.tags: forwarded
  #- add_cloud_metadata: ~
  #- add_docker_metadata: ~
  #- add_kubernetes_metadata: ~
  - decode_json_fields:
      fields: ["message"]
      process_array: true
      max_depth: 1
      target: ""
      overwrite_keys: false

```

i'am not sure where its going wrong but.......with this setup i can see all my log message is under a single field

need some help on this !!!

---

<div class="post-metadata">

**Author:** ![sudhagar\_ramesh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sudhagar_ramesh/32/105673_2.png) [@sudhagar\_ramesh](https://discuss.elastic.co/u/sudhagar_ramesh)\
**Post date:** [August 30, 2022, 4:37am UTC](https://discuss.elastic.co/t/need-to-parse-a-nested-json-message-in-logstash/312979/2 "2022-08-30T04:37:03Z")

</div>

Hello @pavanKumar2K

Could try the below code , this will store the decoded format of json in "LogData" field.

```auto
filter 
{ 

json {
  source => "message"
  target => "LogData"
  skip_on_invalid_json => true
}

}

```

Keep Posted !!! Thanks !!!

---

<div class="post-metadata">

**Author:** ![pavanKumar2K](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pavankumar2k/32/110025_2.png) [@pavanKumar2K](https://discuss.elastic.co/u/pavanKumar2K)\
**Post date:** [September 20, 2022, 7:29am UTC](https://discuss.elastic.co/t/need-to-parse-a-nested-json-message-in-logstash/312979/3 "2022-09-20T07:29:00Z")

</div>

Hi @sudhagar_ramesh , thanks for the reply ........... it works.

But now i have one more issue where if i want to process some complex json objects/data  
elasticserach is considering few values with its default datatypes.

Ex - i am passing date as string - "date":"12-08-2022" , but its being conidered as date format and gives me a mapping parser exception

i am not sure , where i need to make changes for this.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 18, 2022, 7:29am UTC](https://discuss.elastic.co/t/need-to-parse-a-nested-json-message-in-logstash/312979/4 "2022-10-18T07:29:03Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.