Rollover doesn't actually roll in the Python Elasticseach API

[mehdi-gital (Mehdi)]

It's my first time trying to create a rollover and I've read a bunch of posts with similar topics but nothing quite answers mine.

from elasticsearch import Elasticsearch, helpers

es = Elasticsearch([{'host': 'localhost', 'port': 9200}])

es.indices.delete(index='myind-000001', ignore=[400, 404])

my_idx_settings = {
    "template": "myind*", 
    "settings": {"index" : {
        "number_of_shards" : 3, "number_of_replicas": 1}},
    "aliases": {"myind_write": {}},
    "mappings": {
            'mytype': {
            "properties": {
                "vs": {
                    "type": "nested",
                    "properties": {
                        "par": {"type": "keyword"},
                        "value": {"type": "float"}
                    }
                },
                "timestamp": {
                    "type": "date",
                    "format": "yyyy-MM-dd'T'HH:mm:ss"
                    }
            }
        }
    }
}

es.indices.put_template("bm_temp", my_idx_settings)

es.indices.create(index = 'myind-000001')  
#es.indices.put_mapping(index='myind-000001', doc_type='mytype', body=mymap) # don't need this. 

es.indices.put_alias(index=['myind-000001'], name = 'myind_write' )
roll_body = '{"conditions": {"max_docs":  5}}'
es.indices.rollover(alias=alias, body=roll_body)

data = []

for i in range(1,42):
    obj = {}
    obj['_type'] = 'mytype'
    obj['vs'] = [{'par': 'HR', 'val': i}]
    obj['_index'] = 'myind-000001'
    data.append(obj)

data = iter(data)

helpers.bulk(es, data)

This code works except that everything gets ingested into the original index and no rolling takes place.
What am I doing wrong?

Thank you!

[Christian_Dahlqvist]

The rollover operation checks if the rollover condition is met. Only off this is the case will it initiate a rollover. As it looks like you are inserting data after the rollover operation, the condition is not met when you run the operation.

[mehdi-gital (Mehdi)]

@Christian_Dahlqvist Thanks for the comment. I thought I was following the same logic since I have 41 documents and I'm saying max_docs=5 and because my alias and my mapping are linked, I expected the ingestion to follow the pattern determined by the rollover. So, I still don't see how my condition isn't met. Is this somehow related to the order that things appear or that rollovers don't mix with helpers.bulk at all? Any further help would be appreciated.

[Christian_Dahlqvist]

Call es.indices.rollover(alias=alias, body=roll_body) again after the bulk request, and that should initiate a rollover. The check is performed at the point when you make this call, so it is not automatic.

[mehdi-gital (Mehdi)]

Thanks so much for the hint @Christian_Dahlqvist! Although, I'm sad to say that I did try that without success as of yet. One of my colleagues who had worked on this project in the past told me that he had a similar issue and ended up defining the rollover inside Kibana once and then used the Python client for the rest.