# JSON filter swallowing events with no errors

**URL:** <https://discuss.elastic.co/t/json-filter-swallowing-events-with-no-errors/272221>\
**Category:** Logstash\
**Created:** [May 5, 2021, 7:16pm UTC](https://discuss.elastic.co/t/json-filter-swallowing-events-with-no-errors/272221 "2021-05-05T19:16:21Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![brian\_m](https://avatars.discourse-cdn.com/v4/letter/b/848f3c/32.png) [@brian\_m](https://discuss.elastic.co/u/brian_m)\
**Post date:** [May 5, 2021, 7:16pm UTC](https://discuss.elastic.co/t/json-filter-swallowing-events-with-no-errors/272221/1 "2021-05-05T19:16:22Z")

</div>

Hello,

I'm trying to do some testing on ingesting JSON data, and having trouble with the JSON filter. My pipeline configuration is below (output filter has been modified to remove credentials and IPs)

```auto
input {
  http {}
}

filter {
  json {
    source => "message"
    remove_field => ["message"]
  }
  mutate {
    remove_field => ["headers"]
  }
}

output {
  elasticsearch {
    hosts => ["http://localhost:9200"]
    index => ["test-index"]
    user => "user"
    password => "password"
  }
}

```

The issue I'm having is that if I comment out the JSON filter (so input, mutate, and output only) it works fine. I get events in Elastic and they look good (with the exception of being a giant block of text in the message field). With the JSON filter uncommented... I get nothing. No errors in Logstash, no errors in Elasticsearch, and no events in Elastic. If I look in Pipeline Stack Monitoring in Kibana, it seems to show events going through, but nothing shows up.

If I comment out the JSON filter again and re-send the data, it works again.

edit: I also found out that if I specify a target in the JSON filter, it works.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 5, 2021, 9:33pm UTC](https://discuss.elastic.co/t/json-filter-swallowing-events-with-no-errors/272221/2 "2021-05-05T21:33:21Z")

</div>

That suggests to me that you are getting mapping exceptions for some of the fields that the json filter parses. The elasticsearch filter logs those. What does logstash log?

---

<div class="post-metadata">

**Author:** ![brian\_m](https://avatars.discourse-cdn.com/v4/letter/b/848f3c/32.png) [@brian\_m](https://discuss.elastic.co/u/brian_m)\
**Post date:** [May 6, 2021, 1:14pm UTC](https://discuss.elastic.co/t/json-filter-swallowing-events-with-no-errors/272221/3 "2021-05-06T13:14:45Z")

</div>

I watched both the Logstash log and the ES log (at least I thought I did) specifically looking for mapping errors and didn't see them. I just removed the index template entirely (which only had mappings in it) and it worked. So apparently I missed something somewhere. Thank you!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 3, 2021, 1:15pm UTC](https://discuss.elastic.co/t/json-filter-swallowing-events-with-no-errors/272221/4 "2021-06-03T13:15:36Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.