I am using curator version 5.8.1. I plan to replace rundeck with curator at my end. I am using curator_cli to operate. I have curator.yml that works for me.
---
client:
hosts:
- https://###EDITED###.us-east-1.es.amazonaws.com
url_prefix:
use_ssl: False
certificate:
client_cert:
client_key:
ssl_no_validate: False
http_auth:
timeout: 30
master_only: False
logging:
loglevel: DEBUG
logfile: /home/skarpe/logs/curator/curator.log
logformat: default
blacklist: []
I have successfully managed to create new index with following create index 'action.yml' file.
---
actions:
1:
action: create_index
description: Create the index associated with write alias.
options:
name: "<conlogtest-{now{YYYY-MM-dd-HH-mm}}>"
continue_if_exception: True
disable_action: false
extra_settings:
index.number_of_shards: 3
index.number_of_replicas: 1
Now when i tried rollover action i do not get any error but still dont see any new index replacing old one based on condition. Please check the rollover 'action.yml' file for rollover operation.
---
actions:
1:
action: rollover
options:
name: conlogtest
new_index: '<conlogtest-{now{YYYY-MM-dd-HH-mm}}>'
continue_if_exception: True
conditions:
max_age: 1m
extra_settings:
index.number_of_shards: 3
index.number_of_replicas: 1
Last few lines of logs of rollover operation.
2020-09-01 08:19:31,965 INFO curator.cli run:148 Preparing Action ID: 1, "rollover"
2020-09-01 08:19:31,965 INFO curator.cli run:162 Creating client object and testing connection
2020-09-01 08:19:31,965 DEBUG curator.utils get_client:809 kwargs = {'hosts': ['https://###EDITED###.us-east-1.es.amazonaws.com'], 'use_ssl': False, 'ssl_no_validate': False, 'master_only': False, 'port': 9200, 'aws_key': None, 'url_prefix': '', 'aws_sign_request': False, 'aws_token': None, 'http_auth': None, 'client_key': None, 'client_cert': None, 'certificate': None, 'aws_secret_key': None, 'timeout': 30}
2020-09-01 08:19:31,965 DEBUG curator.utils get_client:871 Checking for AWS settings
2020-09-01 08:19:31,968 DEBUG curator.utils get_client:886 "requests_aws4auth" module present, but not used.
2020-09-01 08:19:31,968 INFO curator.utils get_client:903 Instantiating client object
2020-09-01 08:19:31,969 INFO curator.utils get_client:906 Testing client connectivity
2020-09-01 08:19:31,970 DEBUG urllib3.connectionpool _new_conn:813 Starting new HTTPS connection (1): ###EDITED###.us-east-1.es.amazonaws.com:443
2020-09-01 08:19:31,990 DEBUG urllib3.connectionpool _make_request:393 https://###EDITED###.us-east-1.es.amazonaws.com:443 "GET / HTTP/1.1" 200 502
2020-09-01 08:19:31,990 INFO elasticsearch log_request_success:121 GET https://###EDITED###.us-east-1.es.amazonaws.com:443/ [status:200 request:0.021s]
2020-09-01 08:19:31,990 DEBUG elasticsearch log_request_success:123 > None
2020-09-01 08:19:31,991 DEBUG elasticsearch log_request_success:124 < {
"name" : "ct-f2vf",
"cluster_name" : "###EDITED###",
"cluster_uuid" : "###EDITED###",
"version" : {
"number" : "6.8.0",
"build_flavor" : "oss",
"build_type" : "tar",
"build_hash" : "dadef69",
"build_date" : "2020-04-28T01:07:12.310753Z",
"build_snapshot" : false,
"lucene_version" : "7.7.0",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"
}
2020-09-01 08:19:31,991 DEBUG curator.utils get_client:907 Cluster info: {'name': '###EDITED###', 'cluster_name': '###EDITED###', 'cluster_uuid': '###EDITED###', 'version': {'number': '6.8.0', 'build_flavor': 'oss', 'build_type': 'tar', 'build_hash': '###EDITED###', 'build_date': '2020-04-28T01:07:12.310753Z', 'build_snapshot': False, 'lucene_version': '7.7.0', 'minimum_wire_compatibility_version': '5.6.0', 'minimum_index_compatibility_version': '5.0.0'}, 'tagline': 'You Know, for Search'}
2020-09-01 08:19:31,991 INFO curator.utils get_client:908 Successfully created Elasticsearch client object with provided settings
2020-09-01 08:19:31,991 DEBUG curator.utils get_client:932 Checking Elasticsearch endpoint version...
2020-09-01 08:19:31,995 DEBUG urllib3.connectionpool _make_request:393 https://###EDITED###.us-east-1.es.amazonaws.com:443 "GET / HTTP/1.1" 200 502
2020-09-01 08:19:31,996 INFO elasticsearch log_request_success:121 GET https://###EDITED###.us-east-1.es.amazonaws.com:443/ [status:200 request:0.005s]
2020-09-01 08:19:31,996 DEBUG elasticsearch log_request_success:123 > None
2020-09-01 08:19:31,996 DEBUG elasticsearch log_request_success:124 < {
"name" : "p-Nixzh",
"cluster_name" : "###EDITED###:ops-conlog-es",
"cluster_uuid" : "###EDITED###",
"version" : {
"number" : "6.8.0",
"build_flavor" : "oss",
"build_type" : "tar",
"build_hash" : "###EDITED###",
"build_date" : "2020-04-28T01:07:12.310753Z",
"build_snapshot" : false,
"lucene_version" : "7.7.0",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"
}
2020-09-01 08:19:31,996 DEBUG curator.utils check_version:693 Detected Elasticsearch version 6.8.0
2020-09-01 08:19:31,996 DEBUG curator.utils get_client:951 Not verifying local master status (master_only: false)
2020-09-01 08:19:32,015 DEBUG urllib3.connectionpool _make_request:393 https://###EDITED###.us-east-1.es.amazonaws.com:443 "GET /conlogtest/_alias HTTP/1.1" 404 363
2020-09-01 08:19:32,015 WARNING elasticsearch log_request_fail:149 GET https://###EDITED###.us-east-1.es.amazonaws.com:443/conlogtest/_alias [status:404 request:0.019s]
2020-09-01 08:19:32,015 DEBUG elasticsearch log_request_fail:160 > None
2020-09-01 08:19:32,016 DEBUG elasticsearch log_request_fail:165 < {"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index","index_uuid":"_na_","resource.type":"index_or_alias","resource.id":"conlogtest","index":"conlogtest"}],"type":"index_not_found_exception","reason":"no such index","index_uuid":"_na_","resource.type":"index_or_alias","resource.id":"conlogtest","index":"conlogtest"},"status":404}
Note that in above log "GET /conlogtest/_alias" gives me 404. When i check alias for existing index via kibana using "GET /conlogtest-2020-09-01-08-19/_alias" i get below result showing alias as 'conlogtest-2020' and not 'conlogtest'
{ "conlogtest-2020-09-01-08-19" : { "aliases" : { "conlog-2020" : { } } } }. I wonder why is it so. But my main question remains why createindex works for me and rollover does not? Please let me know if any additional info required.
Thanks in advance.