How to automate the Rollover index API in AWS Elastic search

Elastic Stack Elasticsearch
1

Hi,

I am very new to elastic search. We are using ES to store messages coming from our App. Here is how my index looks like
PUT message_history_index
{
"mappings":{
"notification_history" : {
"dynamic": "strict",
"properties":{
"account_id":{
"type": "integer"
},
"user_id": {
"type": "integer"
},
"created_when": {
"type": "date",
"format": "date_time"
}

        }
    }
}

}
I want to create monthly indexes and I am trying to automate this. So that at the end of every month there will be automated process to create an index.So all documents created this month will be in message_history_index-2019.02-1

I created a template to create time series index and the used the rollover api to create to do this

PUT _template/message_history_index_template
{
"template": "message_history_index-*",
"settings": {
"number_of_shards": 5,
"number_of_replicas": 0
},
"aliases": {
"search_message_history": {}
}
}

POST active_message_history/_rollover
{
"conditions": {
"max_age": "30d"
}
}

But my problem is that now ill have to manually execute the rollover after every 30 days for it to create the new index.
Is there any way of automating the rollover api in AWS?
How do I create a cron job in AWS to automate this rollover process?

Thanks in advance

2

In Elasticsearch 6.6 you have now a UI and tools to automate that from Kibana. See Management -> Elasticsearch -> Index Lifecycle Management

image
image.jpg2014×1216 232 KB

Note that this is available in https://www.elastic.co/cloud and https://aws.amazon.com/marketplace/pp/B01N6YCISK ?

Cloud by elastic is one way to have access to all features, all managed by us. Think about what is there yet like Security, Monitoring, Reporting, SQL, Canvas, APM, Logs UI, Infra UI and what is coming next :slight_smile: ...

If you are using Elasticsearch AWS Service, you won't have access to this most likely.

3

Unfortunetly I use ElasticSearch 6.2 on AWS :sweat: Is there any other way to achieve this?

4

You can probably use Curator and call it from a crontab ?
So start a EC2 instance with this crontab or may be a lambda can do this?

6

I don't. Sorry.

Again you should really consider cloud.elastic.co. It runs on AWS or GCP. So practically it should be easy to switch to it.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.