Hi, we normally dont want elastic to create index on insert, so we turned action.auto_create_index = false,
But today we found out that if this option is false index rollover will not work (no rollover at midnight as configured) as no new index was created, are we correct to infer this has an influence on "rollover"?
I think there might be some confusion here - there's no such thing as a "rollover policy". The Rollover API is one-shot: it only checks the conditions and, if one matches, rollover the index at the time you make the API call. If you want to periodically roll over an index, you'll have to have a script periodically call the Rollover API.
You can use Curator to do this, or just a cron job with a curl call.
so the rollover (potentially) happens at the time the of issueing the command POST/<index>/_rollover, But to be honest it still stops after the 6h timelimit, so i am afraid there must be some kind of check maybe
I issued this command at 19:00 last nigh
Thanks for your answer and correcting my persception of the rollover process
We do see that events are still inserted into the index and no new one was created
I issed this command last night at 19:00 hours, (see below POST command)
what does max_age in my new understanding mean, the age of the index (since creation) or the age of the oldest data
also we see that exactly 6 hours later, the monitoring (the "heart" icon in kibana) does not have any metrics on the said indexes. (Everything is working as it should though, as i see data flowing into the system);
I get the feeling the 6h age (issued at 19:00) and the lack of monitoring info (around 01:00) exactly 6 hours later are related. Is this so?
I solved the "monitoring doesnt work" problem, i explicitly turned off auto_create_index
As soon as i turned it on, monitoring began working again, i assume monitoring needs to create new indexes to store metrics.
Regarding your question, max_age refers to the index age, not the oldest data - if you created an index 10 days ago but never indexed any data into it, it would still roll over if you called rollover with a condition of max_age: 10d.
Regarding the monitoring stopping at 01:00, yes, I believe monitoring works by setting up an index template and just indexing docs into the new index when the day switches over, so that makes sense. However, I would expect it to switch over at midnight - do you happen to be located in a time zone 1 hour different from your cluster? It could be the case that the cluster enters the new day (and therefore switches to the new daily index) an hour later than you do, and Kibana will display data in your browser's time zone.