# Log Events/Records that Cause Exceptions

**URL:** https://discuss.elastic.co/t/log-events-records-that-cause-exceptions/94501
**Category:** Logstash
**Created:** [July 25, 2017, 3:00pm UTC](https://discuss.elastic.co/t/log-events-records-that-cause-exceptions/94501 "2017-07-25T15:00:52Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![swsw](https://avatars.discourse-cdn.com/v4/letter/s/c77e96/32.png) [@swsw](https://discuss.elastic.co/u/swsw)
#### Post date: [July 25, 2017, 3:00pm UTC](https://discuss.elastic.co/t/log-events-records-that-cause-exceptions/94501/1 "2017-07-25T15:00:52Z")

</div>

Hello,

How do I log the events/records that come into Logstash that cause exceptions (specifically illegal\_argument\_exception when putting data into Elasticsearch from Logstash).

Situation: I am using Logstash and Elasticsearch 5.3. I have data coming in that I don't have much control over. I have a schema set on the index that Logstash is putting the data into. Sometimes data comes in that doesn't match the schema. I get the following record in the Logstash log:

```auto
[2017-07-24T19:52:38,086][WARN][logstash.outputs.elasticsearch] Failed action. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"active-legacy", :_type=>"type", :_routing=>nil}, 2017-07-24T22:56:54.076Z xxxx %{message}], :response=>{"index"=>{"_index"=>"index-legacy", "_type"=>"type", "_id"=>"AV13BGKU0yA-YGydx47G", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [data_model.fields.utc_time]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: \"2017-07-24 22:56:54.076\" is malformed at \" 22:56:54.076\""}}}}}

```

What I would like is an option to log the records that came in that led to that exception, so that as I tweak my logstash config, I can test to make sure that my changes work.

Something like:

```auto
output {
    elasticsearch {
        hosts => ...
        index => ....
        log_on_failure => "/tmp/es_debug.log"
    }
}

```

Or if instead there was a global logstash option that logs records when exceptions occur?

Does anything like this exist?

Thanks

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [July 26, 2017, 8:02am UTC](https://discuss.elastic.co/t/log-events-records-that-cause-exceptions/94501/2 "2017-07-26T08:02:30Z")

</div>

Have you look at the dead letter queue feature (introduced in 5.5): [https://www.elastic.co/guide/en/logstash/current/dead-letter-queues.html](https://www.elastic.co/guide/en/logstash/current/dead-letter-queues.html)

---

<div class="post-metadata">

### Author: ![swsw](https://avatars.discourse-cdn.com/v4/letter/s/c77e96/32.png) [@swsw](https://discuss.elastic.co/u/swsw)
#### Post date: [July 31, 2017, 4:58pm UTC](https://discuss.elastic.co/t/log-events-records-that-cause-exceptions/94501/3 "2017-07-31T16:58:50Z")

</div>

That's exactly wat I need. Will have to look into upgrading =)

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 28, 2017, 4:59pm UTC](https://discuss.elastic.co/t/log-events-records-that-cause-exceptions/94501/4 "2017-08-28T16:59:16Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
