# Make Logstash drop documents on 403
**URL:** https://discuss.elastic.co/t/make-logstash-drop-documents-on-403/149977
**Category:** Logstash
**Created:** [September 26, 2018, 10:06am UTC](https://discuss.elastic.co/t/make-logstash-drop-documents-on-403/149977 "2018-09-26T10:06:15Z")
**Posts on this page:** 10
**Page:** 1
### Author:  [@programagor](https://discuss.elastic.co/u/programagor)
#### Post date: [September 26, 2018, 10:06am UTC](https://discuss.elastic.co/t/make-logstash-drop-documents-on-403/149977/1 "2018-09-26T10:06:16Z")
Greetings
Recently I started using forcemerge on my old indices. However, I found out that occasionally, Logstash writes into the older indices, increasing the segment count, so the curator has to merge them again on the next day. To prevent this, I now switch older indices to read-only just before merging.
However, now when I look at the Logstash logs, there is a lot of entries like
```
[2018-09-26T11:59:47,219][INFO][logstash.outputs.elasticsearch] retrying failed action with response code: 403 ({"type"=>"cluster_block_exception", "reason"=>"blocked by: [FORBIDDEN/8/index write (api)];"})
```
Is there a way to tell Logstash to drop the documents which receive 403?
Thanks
---
### Author:  [@programagor](https://discuss.elastic.co/u/programagor)
#### Post date: [September 26, 2018, 12:23pm UTC](https://discuss.elastic.co/t/make-logstash-drop-documents-on-403/149977/2 "2018-09-26T12:23:18Z")
Alternatively, is there a way to drop documents already in the Logstash queue with the `@timestamp` field older than 1 day?
---
### Author:  [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)
#### Post date: [September 26, 2018, 1:53pm UTC](https://discuss.elastic.co/t/make-logstash-drop-documents-on-403/149977/3 "2018-09-26T13:53:39Z")
You're going to want to go with the alternative, because Logstash will never just drop documents which yield a 403 error.
There are likely a few timestamp comparison examples here in the discussion forums. That's really all you should need.
---
### Author:  [@programagor](https://discuss.elastic.co/u/programagor)
#### Post date: [September 26, 2018, 2:39pm UTC](https://discuss.elastic.co/t/make-logstash-drop-documents-on-403/149977/4 "2018-09-26T14:39:08Z")
Thanks for the information. Should a feature request be submitted to Logstash to drop documents on 403, or at least place them into DLQ, instead of polluting the output queue? I don't think that it's too specific for my scenario, as the only way to fix the 403 is to either give Logstash user correct privileges, or make the relevant indices writable. Neither of these things is resolved by simply retrying, so placing these documents into DLQ seems reasonable to me.
---
### Author:  [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)
#### Post date: [September 26, 2018, 2:58pm UTC](https://discuss.elastic.co/t/make-logstash-drop-documents-on-403/149977/5 "2018-09-26T14:58:27Z")
[Error 403: "The server understood the request, but is refusing to fulfill it. Authorization will not help and the request SHOULD NOT be repeated."](https://en.wikipedia.org/wiki/HTTP_403).
It's not about privileges. It's that Elasticsearch is flat out refusing to do anything with the message.
Maybe it is a good idea to enable DLQ for this. Maybe it already is. Have you checked? The DLQ feature in Logstash has to be enabled. It isn't on by default.
---
### Author:  [@programagor](https://discuss.elastic.co/u/programagor)
#### Post date: [September 26, 2018, 3:00pm UTC](https://discuss.elastic.co/t/make-logstash-drop-documents-on-403/149977/6 "2018-09-26T15:00:12Z")
I have enabled the DLQ, but it wasn't used in this case.
And yes, the article you linked confirms that the same request (in this case writing to an old index) should not be re-attempted after 403
---
### Author:  [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)
#### Post date: [September 26, 2018, 3:01pm UTC](https://discuss.elastic.co/t/make-logstash-drop-documents-on-403/149977/7 "2018-09-26T15:01:20Z")
Sounds like a good feature request, then.
---
### Author:  [@programagor](https://discuss.elastic.co/u/programagor)
#### Post date: [October 12, 2018, 6:35am UTC](https://discuss.elastic.co/t/make-logstash-drop-documents-on-403/149977/8 "2018-10-12T06:35:09Z")
> [@theuntergeek](#):
>
> There are likely a few timestamp comparison examples here in the discussion forums. That's really all you should need.
Could you please point me at one? I seem unable to find any that would fit my situation. So I asked a new question, but no one is answering that.
> [@Drop old messages using Ruby filter](https://discuss.elastic.co/t/drop-old-messages-using-ruby-filter/151990):
>
> Greetings Occasionally, my logstash receives a message from few days back, and logstash tries to write it into an index which was already marked as readonly and forcemerged. Logstash then receives a 403 error from Elasticsearch, and instead of dropping the message or placing it into DLQ, it keeps retrying it. Eventually, these undeliverable messages clog up the output queue completely, and no valid events are emitted afterwards. I submitted a bug report already, but there seems to be no activ…
---
### Author:  [@programagor](https://discuss.elastic.co/u/programagor)
#### Post date: [October 19, 2018, 5:24am UTC](https://discuss.elastic.co/t/make-logstash-drop-documents-on-403/149977/9 "2018-10-19T05:24:55Z")
I found a way:
```auto
ruby {
init => "require 'time'"
code => 'if LogStash::Timestamp.new(event.get("@timestamp")+432000) < ( LogStash::Timestamp.now)
event.cancel
end'
}
```
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 16, 2018, 5:24am UTC](https://discuss.elastic.co/t/make-logstash-drop-documents-on-403/149977/10 "2018-11-16T05:24:59Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.