Logstash stops processing after forbidden index write

Elastic Stack Logstash
1

Hello,

our logstash stops processing all logs after a few FORBIDDEN/8/index write errors. I guess those are documents that have an old timestamp and the index for that timerange is already frozen.
I believe we had a similar error a few month back and the only way we found out which index / time was meant was by snooping and find the packets related to those error messages. We then set the index from ready_only to write again and restarted logstash and everything was fine.

Neither in logstash nor elasticsearch are entries that indicate which index is causing this.
Is there an easier / more comfortable way to find out which index is the root cause for this?

Thanks and best regards!

2

While it would have been useful for analysis, we are now moving forward and add config so that we use the system time for index creation not the value from @timestamp.

ruby {
code => 'event.set("[@metadata][now]", Time.now.strftime("%Y.%m.%d"))'
}

and

  index => "logstash-%{[@metadata][tmplevel]}-%{[@metadata][now]}"
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.