Logstash stops processing after forbidden index write

Hello,

our logstash stops processing all logs after a few FORBIDDEN/8/index write errors. I guess those are documents that have an old timestamp and the index for that timerange is already frozen.
I believe we had a similar error a few month back and the only way we found out which index / time was meant was by snooping and find the packets related to those error messages. We then set the index from ready_only to write again and restarted logstash and everything was fine.

Neither in logstash nor elasticsearch are entries that indicate which index is causing this.
Is there an easier / more comfortable way to find out which index is the root cause for this?

Thanks and best regards!

While it would have been useful for analysis, we are now moving forward and add config so that we use the system time for index creation not the value from @timestamp.

ruby {
code => 'event.set("[@metadata][now]", Time.now.strftime("%Y.%m.%d"))'
}

and

  index => "logstash-%{[@metadata][tmplevel]}-%{[@metadata][now]}"

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.