How to automate the Rollover index API in AWS Elastic search

Hi,

I am very new to elastic search. We are using ES to store messages coming from our App. Here is how my index looks like
PUT message_history_index
{
"mappings":{
"notification_history" : {
"dynamic": "strict",
"properties":{
"account_id":{
"type": "integer"
},
"user_id": {
"type": "integer"
},
"created_when": {
"type": "date",
"format": "date_time"
}

        }
    }
}

}
I want to create monthly indexes and I am trying to automate this. So that at the end of every month there will be automated process to create an index.So all documents created this month will be in message_history_index-2019.02-1

I created a template to create time series index and the used the rollover api to create to do this

PUT _template/message_history_index_template
{
"template": "message_history_index-*",
"settings": {
"number_of_shards": 5,
"number_of_replicas": 0
},
"aliases": {
"search_message_history": {}
}
}

POST active_message_history/_rollover
{
"conditions": {
"max_age": "30d"
}
}

But my problem is that now ill have to manually execute the rollover after every 30 days for it to create the new index.
Is there any way of automating the rollover api in AWS?
How do I create a cron job in AWS to automate this rollover process?

Thanks in advance

In Elasticsearch 6.6 you have now a UI and tools to automate that from Kibana. See Management -> Elasticsearch -> Index Lifecycle Management

Note that this is available in https://www.elastic.co/cloud and https://aws.amazon.com/marketplace/pp/B01N6YCISK ?

Cloud by elastic is one way to have access to all features, all managed by us. Think about what is there yet like Security, Monitoring, Reporting, SQL, Canvas, APM, Logs UI, Infra UI and what is coming next :slight_smile: ...

If you are using Elasticsearch AWS Service, you won't have access to this most likely.

Unfortunetly I use ElasticSearch 6.2 on AWS :sweat: Is there any other way to achieve this?

You can probably use Curator and call it from a crontab ?
So start a EC2 instance with this crontab or may be a lambda can do this?

Thanks for the quick replies, I was thinking of doing the same. But I am struggling to create a lambda that uses curator to do that. Do you have any code snippets for that?

Thanks,
Seena

I don't. Sorry.

Again you should really consider cloud.elastic.co. It runs on AWS or GCP. So practically it should be easy to switch to it.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.