Rollover index creation failed

[jacobot]

Hi, we normally dont want elastic to create index on insert, so we turned action.auto_create_index = false,
But today we found out that if this option is false index rollover will not work (no rollover at midnight as configured) as no new index was created, are we correct to infer this has an influence on "rollover"?

PUT /_cluster/settings
{
   "persistent": {
      "action.auto_create_index": false
   }
}

[gbrown (Gordon Brown)]

That setting should have no impact on rollover - I just tested it to make 100% sure. What conditions are you using for the rollover call?

[jacobot]

Thanks Gorden
Is there a way to get the current existing rollover policies from the index or its alias? (just to be sure)

I was reading this https://www.elastic.co/guide/en/elasticsearch/reference/master/indices-rollover-index.html

But it only mentions how to set a rollover policy,

[jacobot]

POST /<indexname obfuscated>/_rollover 
{
 "conditions": {
    "max_age": "1d", 
    "max_size": "10gb" 
  } 
}

1d or 10gb

But how do i check, since i issued several rollover conditions, in the past, but to be sure i want to know what is currently is in effect.

Jacob

[gbrown (Gordon Brown)]

I think there might be some confusion here - there's no such thing as a "rollover policy". The Rollover API is one-shot: it only checks the conditions and, if one matches, rollover the index at the time you make the API call. If you want to periodically roll over an index, you'll have to have a script periodically call the Rollover API.

You can use Curator to do this, or just a cron job with a curl call.

[jacobot]

aaah,
so the rollover (potentially) happens at the time the of issueing the command POST/<index>/_rollover, But to be honest it still stops after the 6h timelimit, so i am afraid there must be some kind of check maybe

I issued this command at 19:00 last nigh

Hi Gordon
Thanks for your answer and correcting my persception of the rollover process
We do see that events are still inserted into the index and no new one was created

I issed this command last night at 19:00 hours, (see below POST command)

1. what does max_age in my new understanding mean, the age of the index (since creation) or the age of the oldest data

2. also we see that exactly 6 hours later, the monitoring (the "heart" icon in kibana) does not have any metrics on the said indexes. (Everything is working as it should though, as i see data flowing into the system);

3. I get the feeling the 6h age (issued at 19:00) and the lack of monitoring info (around 01:00) exactly 6 hours later are related. Is this so?

POST /prod_eds_messages/_rollover
{
  "conditions": {
    "max_age":   "6h",
    "max_size": "10gb"
  }
}

Kibana snippet

[jacobot]

update:
I solved the "monitoring doesnt work" problem, i explicitly turned off auto_create_index
As soon as i turned it on, monitoring began working again, i assume monitoring needs to create new indexes to store metrics.

[gbrown (Gordon Brown)]

Regarding your question, max_age refers to the index age, not the oldest data - if you created an index 10 days ago but never indexed any data into it, it would still roll over if you called rollover with a condition of max_age: 10d.

Regarding the monitoring stopping at 01:00, yes, I believe monitoring works by setting up an index template and just indexing docs into the new index when the day switches over, so that makes sense. However, I would expect it to switch over at midnight - do you happen to be located in a time zone 1 hour different from your cluster? It could be the case that the cluster enters the new day (and therefore switches to the new daily index) an hour later than you do, and Kibana will display data in your browser's time zone.

[jacobot]

@gbrown

setting

yes my timezone is CET (GMT+1), i am not sure of the timezone of the host the elastic is on, its somewhere in Asia, ))
In any case I am leaving the auto_create_index to true from now on)