# Json\_encode returns null

**URL:** <https://discuss.elastic.co/t/json-encode-returns-null/114728>\
**Category:** Logstash\
**Created:** [January 9, 2018, 3:29pm UTC](https://discuss.elastic.co/t/json-encode-returns-null/114728 "2018-01-09T15:29:04Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sebastian\_Fratini](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sebastian_fratini/32/26392_2.png) [@Sebastian\_Fratini](https://discuss.elastic.co/u/Sebastian_Fratini)\
**Post date:** [January 9, 2018, 3:29pm UTC](https://discuss.elastic.co/t/json-encode-returns-null/114728/1 "2018-01-09T15:29:04Z")

</div>

I am parsing data from the pm2 manager for node and it works fine, however some of my data ends in a text field instead of an structure. This is my logstash configuration:

```
if [fields][type] == "pm2" {
        json {
            source => "message"
            target => "pm2"
        }
        json_encode {
            source => "pm2.message"
            target => "pm2.dataString"
        }
        json {
            source => "pm2.dataString"
            target => "pm2.data"
        }

        date {
            match => ["timestamp", "yyyy-MM-dd HH:mm:ssZ"]
            tag_on_failure => ["_dateparsefailure"]
        }
    }

```

Part of my template:

```
"pm2" : {
            "properties" : {
              "data" : {
                "type" : "object"
              },
              "dataString" : {
                "type" : "text"
              },
              "message" : {
                "type" : "text"
              },

```

Then some example data:

Original message: {"message":"{\n "className": "AppointmentBot",\n "dialog": "query",\n "step": 1,\n "millis": 0,\n "extra": "",\n "type": "logger",\n "level": "info",\n "message": "Query",\n "timestamp": "09-01-2018 15:13:05 +00:00"\n}\n","timestamp":"2018-01-09T15:13:05.310Z","type":"out","process\_id":9,"app\_name":"Chatbot-App-Dev"}

pm2.message ends up being this. which I don't want, I want every field separate: {  
"className": "AppointmentBot",  
"dialog": "query",  
"step": 1,  
"millis": 0,  
"extra": "",  
"type": "logger",  
"level": "info",  
"message": "Query",  
"timestamp": "09-01-2018 15:13:05 +00:00"  
}

So that is why I am trying to encode the JSON back to string and parse it to the data field which I made an object, but I am getting this that dataString is null.

Any idea? Thanks

---

<div class="post-metadata">

**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\
**Post date:** [January 9, 2018, 4:44pm UTC](https://discuss.elastic.co/t/json-encode-returns-null/114728/2 "2018-01-09T16:44:11Z")

</div>

I got this to work...  
Note: you simply need to apply the json plugin to the message field twice, however if you have to continue with your approach then you should know that nested fields are referenced in this `[a][b][c]` style.

```auto
input {
  generator {
    message => '{"message":"{\n \"className\": \"AppointmentBot\",\n \"dialog\": \"query\",\n \"step\": 1,\n \"millis\": 0,\n \"extra\": \"\",\n \"type\": \"logger\",\n \"level\": \"info\",\n \"message\": \"Query\",\n \"timestamp\": \"09-01-2018 15:13:05 +00:00\"\n }\n ","timestamp":"2018-01-09T15:13:05.310Z","type":"out","process_id":9,"app_name":"Chatbot-App-Dev"}'
    count => 1
  }
}

filter {
  json {
    source => "message"
  }
  if "_jsonparsefailure" not in [tags] {
    json {
      source => "message"
    }
  }
  if ! [Status] {
    mutate {
      update => { "Status" => "Failed" }
    }
  }
}

output {
  stdout {
    codec => rubydebug
  }
}

```

results

```auto
{
        "millis" => 0,
    "@timestamp" => 2018-01-09T16:20:07.127Z,
      "@version" => "1",
     "className" => "AppointmentBot",
      "app_name" => "Chatbot-App-Dev",
         "extra" => "",
          "host" => "Elastics-MacBook-Pro.local",
          "type" => "logger",
        "dialog" => "query",
          "step" => 1,
         "level" => "info",
     "timestamp" => "09-01-2018 15:13:05 +00:00",
      "sequence" => 0,
    "process_id" => 9,
       "message" => "Query"
}

```

---

<div class="post-metadata">

**Author:** ![Sebastian\_Fratini](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sebastian_fratini/32/26392_2.png) [@Sebastian\_Fratini](https://discuss.elastic.co/u/Sebastian_Fratini)\
**Post date:** [January 9, 2018, 6:54pm UTC](https://discuss.elastic.co/t/json-encode-returns-null/114728/3 "2018-01-09T18:54:32Z")

</div>

Excellent! I owe you a beer!

My problem was indeed the [a][b] as the filters were not actually running. I ended up doing this:

```
 json {
    source => "message"
    target => "[pm2]"
    remove_field => "message"
  }
  if "_jsonparsefailure" not in [tags] {
    json {
      source => "[pm2][message]"
      target => "[pm2][data]"
      remove_field => "[pm2][message]"
    }
  }

```

Now, is there a way to use the skip\_on\_invalid\_json and still get the filter to tag the entry if it is invalid? My issue is that sometimes the message property is not a json and while everything works, I don't want to fill the logstash log with warnings.

Thanks again!

---

<div class="post-metadata">

**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\
**Post date:** [January 9, 2018, 6:57pm UTC](https://discuss.elastic.co/t/json-encode-returns-null/114728/4 "2018-01-09T18:57:55Z")

</div>

Just set `skip_on_invalid_json` to true in the json filter config.

```auto
json {
  source => "x"
  skip_on_invalid_json => true
}

```

---

<div class="post-metadata">

**Author:** ![Sebastian\_Fratini](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sebastian_fratini/32/26392_2.png) [@Sebastian\_Fratini](https://discuss.elastic.co/u/Sebastian_Fratini)\
**Post date:** [January 9, 2018, 7:06pm UTC](https://discuss.elastic.co/t/json-encode-returns-null/114728/5 "2018-01-09T19:06:25Z")

</div>

> [@guyboertje](#):
>
> skip\_on\_invalid\_json =\> true

I would have sweared I tried that and it wasn't tagging the entries but I guessed I also played with the fields order and configuration. Everything working now. _tips fedora_

---

<div class="post-metadata">

**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\
**Post date:** [January 9, 2018, 7:07pm UTC](https://discuss.elastic.co/t/json-encode-returns-null/114728/6 "2018-01-09T19:07:56Z")

</div>

Yay, glad to help.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 6, 2018, 7:08pm UTC](https://discuss.elastic.co/t/json-encode-returns-null/114728/7 "2018-02-06T19:08:02Z")

</div>

Closed on February 6, 2018, 7:08pm UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.