I'm using fluentd to send pod logs to ES 8.5
My problem is that each log sent is creating a new index, so i end up having a very large number of indices.
What I'd like to have is one index per day, and keep them for only 7 days.
this is the config i have for the index name:
And what i see in kibana are a lot of indexes created like this:
Another problem is that if i setup in Discover an view for odc-dev-new* i don't see any info there.
I'm not sure what I'm missing in my configuration.
Could you please help with a suggestion?
If you are sending data using fluentd, this issue is related to fluentd.
Something in fluentd config is making it write to indices with the date in the name, you need to check with the fluentd community.
From what you shared, if you have indices with date of
1970-*, you probably have some error parsing the data in epoch.
You have indices with
1970-* in the nane, try to change the time range in discover to show logs from this date.
thank you for the very prompt reply.
this is my config to send the logs to ES:
I do not use fluentd and cannot help further.
As I said, your issue is related on how fluentd is sending data to Elasticsearch, you need to check this issue with the Fluentd community.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.