# JSON format Decode error

**URL:** <https://discuss.elastic.co/t/json-format-decode-error/324652>\
**Category:** Logstash\
**Created:** [February 3, 2023, 1:46pm UTC](https://discuss.elastic.co/t/json-format-decode-error/324652 "2023-02-03T13:46:30Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![djrshn2346](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/djrshn2346/32/108594_2.png) [@djrshn2346](https://discuss.elastic.co/u/djrshn2346)\
**Post date:** [February 3, 2023, 1:46pm UTC](https://discuss.elastic.co/t/json-format-decode-error/324652/1 "2023-02-03T13:46:30Z")

</div>

I am using :

```auto
input {
file {
        id => "my_lt_log"
        path => "/logs/logtransformer.log"
        type => "log"
        start_position => "beginning"
      }
}
filter {
if [type] == "log" {
        mutate {
            remove_field => ["kubernetes"]
        }
        mutate {
          gsub => ["message", "(\W)at(\W)", '\1""\2']
        }
        if [message][metadata][proc_id] {
            mutate {
              add_field => { "[metadata][proc_id]" => "%{[message][metadata][proc_id]}" }
            }
            }
        if "_jsonparsefailure" in [tags] {
          mutate {
            add_field => {
              "logplane" => "adp-app-logs"
              "abc" => "%{[message]}"
            }
            remove_field => ["message", "kubernetes"]
          }
        }
        else {
          mutate {
              rename => {
                "path" => "filename"
              }
              add_field => {
                "def" => "%{[message]}"
                "logplane" => "adp-app-logs"
                "version" => "%{[message][version]}"
                "severity" => "%{[message][severity]}"
                "service_id" => "%{[message][service_id]}"
                "[metadata][container_name]" => "%{[message][metadata][container_name]}"
                "[metadata][node_name]" => "%{[message][metadata][node_name]}"
                "[metadata][namespace]" => "%{[message][metadata][namespace]}"
                "[metadata][pod_name]" => "%{[message][metadata][pod_name]}"
                "[metadata][pod_uid]" => "%{[message][metadata][pod_uid]}"
                "message" => "%{[message][message]}"
                "timestamp" => "%{[message][timestamp]}"
              }
            }
            mutate {
                  remove_field => ["type", "host", "message", "kubernetes"]
              }
            }
      }
}
output {
...
}

```

Output in Elastic Search:

```auto
{
        "_index" : "adp-app-logs-2023.02.03",
        "_type" : "_doc",
        "_id" : "PpiDF4YBCMtUNdxoMJFW",
        "_score" : 0.79323065,
        "_source" : {
          "filename" : "/logs/logtransformer.log",
          "metadata" : {
            "pod_name" : "%{[message][metadata][pod_name]}",
            "container_name" : "%{[message][metadata][container_name]}",
            "node_name" : "%{[message][metadata][node_name]}",
            "namespace" : "%{[message][metadata][namespace]}",
            "pod_uid" : "%{[message][metadata][pod_uid]}"
          },
          "timestamp" : "%{[message][timestamp]}",
          "version" : "%{[message][version]}",
          "@version" : "1",
          "service_id" : "%{[message][service_id]}",
          "def" : "{\"version\": \"1.1.0\", \"timestamp\": \"2023-02-03T13:41:43.034Z\", \"severity\": \"info\", \"service_id\": \"eric-log-transformer\", \"metadata\" : {\"namespace\": \"zyadros\", \"pod_name\": \"eric-log-transformer-7b64896976-s6h5r\", \"node_name\": \"node-10-63-142-135\", \"pod_uid\": \"336c9706-41a9-41c0-b459-2eb4e9f6e2b4\", \"container_name\": \"logtransformer\"}, \"message\": \"Starting pipeline {:pipeline_id=>'opensearch', 'pipeline.workers'=>2, 'pipeline.batch.size'=>2048, 'pipeline.batch.delay'=>50, 'pipeline.max_inflight'=>4096, 'pipeline.sources'=>['/opt/logstash/resource/searchengine.conf'], :thread=>'#<Thread:0x7649ae47 run>'}\"}",
          "@timestamp" : "2023-02-03T13:41:58.044976Z",
          "logplane" : "adp-app-logs",
          "severity" : "%{[message][severity]}"
        }
      }

```

How will i get the values and also decode the message in 'def'?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 3, 2023, 1:46pm UTC](https://discuss.elastic.co/t/json-format-decode-error/324652/2 "2023-02-03T13:46:31Z")

</div>

OpenSearch/OpenDistro are AWS run products and differ from the original Elasticsearch and Kibana products that Elastic builds and maintains. You may need to contact them directly for further assistance.

(This is an automated response from your friendly Elastic bot. Please report this post if you have any suggestions or concerns :elasticheart: )

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [February 3, 2023, 5:12pm UTC](https://discuss.elastic.co/t/json-format-decode-error/324652/3 "2023-02-03T17:12:56Z")

</div>

You should use a json filter to parse the JSON. With your current configuration you will never have a \_jsonparsefailure tag. Also, fields like [message][metadata][proc\_id] are never going to exist if you do not add them.

---

<div class="post-metadata">

**Author:** ![djrshn2346](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/djrshn2346/32/108594_2.png) [@djrshn2346](https://discuss.elastic.co/u/djrshn2346)\
**Post date:** [February 3, 2023, 7:51pm UTC](https://discuss.elastic.co/t/json-format-decode-error/324652/4 "2023-02-03T19:51:49Z")

</div>

I already used json filter earlier but getting multiple errors like 'Error Parsing JSON'. The configuration mentioned above is the correct till now... Can you please give any example?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 3, 2023, 7:52pm UTC](https://discuss.elastic.co/t/json-format-decode-error/324652/5 "2023-03-03T19:52:31Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.