# Logstash json filter's add\_field ignored **URL:** \ **Category:** Logstash\ **Created:** [December 7, 2015, 9:04am UTC](https://discuss.elastic.co/t/logstash-json-filters-add-field-ignored/36504 "2015-12-07T09:04:16Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![vgondil](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vgondil/32/7557_2.png) [@vgondil](https://discuss.elastic.co/u/vgondil)\ **Post date:** [December 7, 2015, 9:04am UTC](https://discuss.elastic.co/t/logstash-json-filters-add-field-ignored/36504/1 "2015-12-07T09:04:16Z")
I'm trying to use the json filter for Logstash 2.1. I find the "add\_field" option is ignored. I had a more complex config earlier, but I stripped it down to test just the add\_field. > input { > generator{ > lines =\> ['{"@message": {"clicked":true}}'] > codec =\> "json" > count =\> 1 > } > } > filter { > json { > source =\> "message" > add\_field =\> { "somefield" =\> "Well hello there!" } > } > } > output { > stdout { codec =\> rubydebug } > } **Console on run:** > E:\ELK\logstash-2.1.0\>bin\logstash agent -f logstash.conf > io/console not supported; tty will not be manipulated > Settings: Default filter workers: 2 > Logstash startup completed > { > "@message" =\> { > "clicked" =\> true > }, > "@version" =\> "1", > "@timestamp" =\> "2015-12-07T08:34:45.508Z", > "host" =\> "IND-PUN-LAP-096", > "sequence" =\> 0 > } Is there something I'm doing wrong here, or is this a bug with 2.1? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [December 7, 2015, 9:28am UTC](https://discuss.elastic.co/t/logstash-json-filters-add-field-ignored/36504/2 "2015-12-07T09:28:24Z")
Your JSON message gets unmarshaled by the json codec in your input, and hence the subsequent json filter won't work since there is no `message` field to parse (so it bails out early and won't trigger `add_field`). ---
**Author:** ![vgondil](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vgondil/32/7557_2.png) [@vgondil](https://discuss.elastic.co/u/vgondil)\ **Post date:** [December 7, 2015, 9:57am UTC](https://discuss.elastic.co/t/logstash-json-filters-add-field-ignored/36504/3 "2015-12-07T09:57:43Z")
Thank you. When I switched from generating to a simple text file with the json, it worked. However, when I move back towards my original pipeline - kafka input, json filter, console output, I once more find the add\_field ineffective. > input { > kafka { > zk\_connect =\> "localhost:2181" > topic\_id =\> "elktest" > } > } > filter { > json { > source =\> "message" > add\_field =\> { "somefield" =\> "Well hello there!" } > } > } > output { > stdout { codec =\> rubydebug } > } My Kafka producer console: > C:\kafka\_2.10-0.8.2.1\>.\bin\windows\kafka-console-producer.bat --broker-list localhost:9092 --topic elktest > [2015-12-07 15:17:59,764] WARN Property topic is not valid (kafka.utils.VerifiableProperties) > {"clicked":true} Logstash console: > E:\ELK\logstash-2.1.0\>bin\logstash agent -f add\_field.conf > io/console not supported; tty will not be manipulated > Settings: Default filter workers: 2 > Logstash startup completed > { > "clicked" =\> true, > "@version" =\> "1", > "@timestamp" =\> "2015-12-07T09:48:06.234Z" > } Am I doing something wrong here? My original configuration needed to extract a nested timestamp from the json consumed from kafka. The input json consumed from Kafka was something like : > { > "userInfo": { > "logTime": "2015-10-06 05:54:53.106", > ... > } > } I needed to extract logTime and assign it to an eventTimestamp. So my filter looked like this: > filter { > json { > source =\> "message" > add\_field =\> { "eventTimestamp" =\> "%{[userInfo][logTime]}" } > } > } Is this incorrect? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [December 7, 2015, 10:18am UTC](https://discuss.elastic.co/t/logstash-json-filters-add-field-ignored/36504/4 "2015-12-07T10:18:24Z")
AFAICT the events don't have a `message`field for the json filter to parse. Apart from `@timestamp` (that Logstash probably added on its own) there's just `clicked`. ---
**Author:** ![vgondil](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vgondil/32/7557_2.png) [@vgondil](https://discuss.elastic.co/u/vgondil)\ **Post date:** [December 7, 2015, 10:47am UTC](https://discuss.elastic.co/t/logstash-json-filters-add-field-ignored/36504/5 "2015-12-07T10:47:24Z")
Sorry, I'm rather new to Logstash. Isn't "message" where the payload ends up by default? Won't the entire json read from Kafka be available in "message"? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [December 7, 2015, 11:28am UTC](https://discuss.elastic.co/t/logstash-json-filters-add-field-ignored/36504/6 "2015-12-07T11:28:47Z")
Not with `codec => "json"` which is the default for the kafka input. Then the JSON payload will be unmarshaled by the input plugin and there will only be a `message` field if the JSON payload defines it. ---
**Author:** ![vgondil](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vgondil/32/7557_2.png) [@vgondil](https://discuss.elastic.co/u/vgondil)\ **Post date:** [December 7, 2015, 11:37am UTC](https://discuss.elastic.co/t/logstash-json-filters-add-field-ignored/36504/7 "2015-12-07T11:37:05Z")
Aaaah! Thanks a ton! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 5:19am UTC](https://discuss.elastic.co/t/logstash-json-filters-add-field-ignored/36504/8 "2017-07-06T05:19:44Z")