# Difference between json codec in input {} vs filter {}?

**URL:** <https://discuss.elastic.co/t/difference-between-json-codec-in-input-vs-filter/77575>\
**Category:** Logstash\
**Created:** [March 6, 2017, 8:43pm UTC](https://discuss.elastic.co/t/difference-between-json-codec-in-input-vs-filter/77575 "2017-03-06T20:43:09Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![ZillaG](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zillag/32/10505_2.png) [@ZillaG](https://discuss.elastic.co/u/ZillaG)\
**Post date:** [March 6, 2017, 8:43pm UTC](https://discuss.elastic.co/t/difference-between-json-codec-in-input-vs-filter/77575/1 "2017-03-06T20:43:10Z")

</div>

I have JSON-ified logs coming into my Logstash server. What's the difference between putting the json codec in

```
input {
  codec => json
}

```

vs.

```
filter {
  json => {
    source => "message"
  }
}
```

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [March 6, 2017, 8:56pm UTC](https://discuss.elastic.co/t/difference-between-json-codec-in-input-vs-filter/77575/2 "2017-03-06T20:56:05Z")

</div>

The json filter and the json codec accomplish the same thing. One point of the json filter is that you can apply it selectively, e.g. for just one of the fields and only under certain conditions. You can probably always use a json filter instead of a json codec but the opposite isn't true.

---

<div class="post-metadata">

**Author:** ![ZillaG](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zillag/32/10505_2.png) [@ZillaG](https://discuss.elastic.co/u/ZillaG)\
**Post date:** [March 6, 2017, 9:03pm UTC](https://discuss.elastic.co/t/difference-between-json-codec-in-input-vs-filter/77575/3 "2017-03-06T21:03:58Z")

</div>

Thanks. I'm having \_jsonparsefailure when I use the json codec so I'm trying the json filter approach.

---

<div class="post-metadata">

**Author:** ![ZillaG](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zillag/32/10505_2.png) [@ZillaG](https://discuss.elastic.co/u/ZillaG)\
**Post date:** [March 6, 2017, 10:02pm UTC](https://discuss.elastic.co/t/difference-between-json-codec-in-input-vs-filter/77575/4 "2017-03-06T22:02:22Z")

</div>

Using the json codec, my fields are automagicaly separated for me, that is, I see them in Kibana. How can I extract my fields using the json filter? My log events look like this.

```
{
  "key1":"value1",
  "key2":"value2",
  "msg":"2017-03-06 INFO [com.company.app] Hello world"
}

```

I tried

```
json => {
  source => "message"
  add_field => {
    "key1" => "%{[message][key1]}"
    "key2" => "%{[message][key2]}"
    "log_msg" => "%{[message][msg]}"
}
```

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [March 7, 2017, 6:12am UTC](https://discuss.elastic.co/t/difference-between-json-codec-in-input-vs-filter/77575/5 "2017-03-07T06:12:45Z")

</div>

Show your full configuration and what your events look like when they leave Logstash. Use a `stdout { codec => rubydebug }` output so we can see exactly what's going on.

---

<div class="post-metadata">

**Author:** ![ZillaG](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zillag/32/10505_2.png) [@ZillaG](https://discuss.elastic.co/u/ZillaG)\
**Post date:** [March 7, 2017, 4:53pm UTC](https://discuss.elastic.co/t/difference-between-json-codec-in-input-vs-filter/77575/6 "2017-03-07T16:53:32Z")

</div>

Here's the JSON log event

```
{
  "key1":"value1",
  "key2":"value2",
  "msg":"2017-03-06 INFO [com.company.app] Hello world"
}

```

Here's my configuration file

```
input {
  udp {
    port => 5555
  }
}

filter {
  mutate {
    strip => "message"
  }
  json {
    source => "message"
    # What do I put here to pick apart the fields in "msg" so my grok filter works?
  }
  grok {
    match => {
      "msg" => {
         "%{TIMESTAMP_ISO8601:logdate}%{SPACE}%\[(?<classname>[^\]]+)\]%{SPACE}%{GREEDYDATA:msgbody}"
      }
    }
  }
}

output {
  stdout {
    codec => rubydebug
  }
}

```

Here's what i see in logstash.stdout

```
{
     "message" => "{\"key1\":\"value1\", \"key2\":\"value2\", \"msg\":\"2017-03-06 INFO [com.company.app] Hello world\"}",
    "@version" => "1",
  "@timestamp" => "2017-03-06",
        "host" => "0:0:0:0:0:0:0:1"
}

```

Here's what I want to see in logstash.stdout, which is what I see when I use the JSON "codec."

```
{
     "message" => "{\"key1\":\"value1\", \"key2\":\"value2\", \"msg\":\"2017-03-06 INFO [com.company.app] Hello world\"}",
        "key1" => "value1",
        "key2" => "value2",
   "classname" => "com.company.app",
    "loglevel" => "INFO",
     "msgbody" => "Hello world", 
    "@version" => "1",
  "@timestamp" => "2017-03-06",
        "host" => "0:0:0:0:0:0:0:1
}
```

---

<div class="post-metadata">

**Author:** ![ZillaG](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zillag/32/10505_2.png) [@ZillaG](https://discuss.elastic.co/u/ZillaG)\
**Post date:** [March 7, 2017, 7:23pm UTC](https://discuss.elastic.co/t/difference-between-json-codec-in-input-vs-filter/77575/7 "2017-03-07T19:23:23Z")

</div>

I think I got it now. [This Stack Overflow post](http://stackoverflow.com/questions/33937936/how-to-parse-json-in-logstash-grok-from-a-text-file-line) gave me ideas. So my filter plugin looks like this now

```
filter {
  json {
    source => "message"
    target => "parsedJson"
  }
  mutate {
    add_field => {
      "key1" => "%{[parsedJson][key1]}"
      "key2" => "%{[parsedJson][key2]}"
      "log_message" => "%{[parsedJson][msg]}"
    }
  }
  grok {
    match => {
      "log_message" => [
           # Grok filters go here.....
      ]
    }
  }
  mutate {
    remove_field => ["message", "parsedJson", "log_message"]
  }
}
```

---

<div class="post-metadata">

**Author:** ![ZillaG](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zillag/32/10505_2.png) [@ZillaG](https://discuss.elastic.co/u/ZillaG)\
**Post date:** [March 7, 2017, 9:22pm UTC](https://discuss.elastic.co/t/difference-between-json-codec-in-input-vs-filter/77575/8 "2017-03-07T21:22:44Z")

</div>

Alas, the json filter gives the same \_jsonparsefailure as the json codec. I should've expected that, though I was hoping otherwise.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 4, 2017, 9:22pm UTC](https://discuss.elastic.co/t/difference-between-json-codec-in-input-vs-filter/77575/9 "2017-04-04T21:22:49Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.