# Failure\_type\_logging\_whitelist not working for document\_missing\_exception type

**URL:** <https://discuss.elastic.co/t/failure-type-logging-whitelist-not-working-for-document-missing-exception-type/164845>\
**Category:** Logstash\
**Created:** [January 18, 2019, 6:33pm UTC](https://discuss.elastic.co/t/failure-type-logging-whitelist-not-working-for-document-missing-exception-type/164845 "2019-01-18T18:33:47Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![jdmcalee](https://avatars.discourse-cdn.com/v4/letter/j/90ced4/32.png) [@jdmcalee](https://discuss.elastic.co/u/jdmcalee)\
**Post date:** [January 18, 2019, 6:33pm UTC](https://discuss.elastic.co/t/failure-type-logging-whitelist-not-working-for-document-missing-exception-type/164845/1 "2019-01-18T18:33:47Z")

</div>

I have a pipeline set up to update documents in elasticsearch if they are found to have been removed from a database. Due to implementation details, I've decided it's easier to issue updates that fail if documents don't exist, rather than ensure documents do exist before updating them. That works fine, but it's logging WARN messages for all documents that don't exist, of course.

So, I configured the failure\_type\_logging\_whitelist option with document\_missing\_exception in order to prevent that:

```
output {
    elasticsearch {
        index => "customers-advtest"
        hosts => ["elsearch"]
        document_id => "%{externalSystem}-%{remoteId}"
        action => update
        script => "ctx._source.status = deactivated"
        failure_type_logging_whitelist => ["document_missing_exception"]
    }
}

```

After doing so, I can see that set in the config:

```
[DEBUG][logstash.outputs.elasticsearch] config LogStash::Outputs::ElasticSearch/@failure_type_logging_whitelist = ["document_missing_exception"]

```

Yet I still get the WARN in the logs:

```
[WARN][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>404, :action=>["update", {:_id=>"adv-37_1001a000197", :_index=>"customers-advtest", :_type=>"doc", :routing=>nil, :_retry_on_conflict=>1}, #<LogStash::Event:0x1a0e94dc>], :response=>{"update"=>{"_index"=>"customers-advtest", "_type"=>"doc", "_id"=>"adv-37_1001a000197", "status"=>404, "error"=>{"type"=>"document_missing_exception", "reason"=>"[doc][adv-37_1001a000197]: document missing", "index_uuid"=>"qhA67Ep3R4a0NVoZLv8kHg", "shard"=>"0", "index"=>"customers-advtest"}}}}

```

What am I missing? Can only certain failure types be whitelisted?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [January 18, 2019, 6:43pm UTC](https://discuss.elastic.co/t/failure-type-logging-whitelist-not-working-for-document-missing-exception-type/164845/2 "2019-01-18T18:43:55Z")

</div>

Looking at the [code](https://github.com/logstash-plugins/logstash-output-elasticsearch/blob/e3fa7e8626e9a63fb366221451313931e918f32a/lib/logstash/outputs/elasticsearch/common.rb#L227), there is no test against the whitelist when it goes through the handle\_dlq\_status branch (i.e. a 400 or 404). One might argue that is a bug.

---

<div class="post-metadata">

**Author:** ![jdmcalee](https://avatars.discourse-cdn.com/v4/letter/j/90ced4/32.png) [@jdmcalee](https://discuss.elastic.co/u/jdmcalee)\
**Post date:** [January 18, 2019, 7:29pm UTC](https://discuss.elastic.co/t/failure-type-logging-whitelist-not-working-for-document-missing-exception-type/164845/3 "2019-01-18T19:29:23Z")

</div>

ah, I see now. the documentation should include this explanation, at least. thanks

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 15, 2019, 7:29pm UTC](https://discuss.elastic.co/t/failure-type-logging-whitelist-not-working-for-document-missing-exception-type/164845/4 "2019-02-15T19:29:37Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.