I have a pair of ES 6.1.1 clusters and have deployed curator 5.4.1 against one of them. Curator and curator_cli commands have no effect. Logging is relatively sparse, however in debug mode I get some output, which I've included along with config files below.
I've disabled ssl validation as I'm using self-signed certs for this instance. I see no errors suggesting this is an issue, but thought it worthy of note for troubleshooting purposes.
Do you have any suggestions for things to explore in troubleshooting this issue?
Thanks.
/etc/elasticsearch/elasticsearch.yml below:
---
cluster:
name: elk_test_cluster
routing:
allocation:
allow_rebalance: always
disk:
watermark:
low: 90%
high: 90%
awareness:
attributes: aws_availability_zone
force:
aws_availability_zone:
values: us-east-1a,us-east-1b
path:
data: "/mnt/elasticsearch"
logs: "/var/log/elasticsearch/"
network:
host:
- _eth0_ # _ec2_
- _local_
bind_host:
- _eth0_ # _ec2_
- _local_
publish_host:
- _eth0_ # _ec2_
- _local_
http:
port: '9200'
xpack:
security:
enabled: true
audit:
enabled: true
http:
ssl:
enabled: true
transport:
ssl:
enabled: true
verification_mode: certificate
authc:
realms:
realm1:
type: native
order: '0'
enabled: true
realm2:
type: file
order: '1'
enabled: true
monitoring:
enabled: true
graph:
enabled: true
watcher:
enabled: true
ssl:
key: "/etc/elasticsearch/x-pack/elasticsearch.key"
certificate: "/etc/elasticsearch/x-pack/elasticsearch.crt"
certificate_authorities:
- "/etc/elasticsearch/x-pack/ca.crt"
notification:
email:
account:
smtp_account:
profile: standard
smtp:
host: localhost
from: nobody@nowhere.com
cloud:
node:
auto_attributes: 'true'
discovery:
zen:
hosts_provider: ec2
ec2:
tag:
efk_cluster_name: elk_test_cluster
host_type: private_ip
availability_zones:
- us-east-1a
- us-east-1b
- us-east-1c
- us-east-1d
- us-east-1e
endpoint: ec2.us-east-1.amazonaws.com
node:
name: ip-XXX-XX-XX-XXX
master: 'true'
data: 'false'
ingest: 'false'
thread_pool:
bulk:
queue_size: '1000'
search:
queue_size: '10000'
/etc/curator/config.yml below:
# Remember, leave a key empty if there is no value. None will be a string,
# not a Python "NoneType"
client:
hosts:
- ["_eth0_", "_local_"]
port: 9200
url_prefix:
use_ssl: True
certificate:
client_cert:
client_key:
ssl_no_validate: True
http_auth: elastic:<PASSWORD>
timeout: 60
master_only: False
logging:
loglevel: DEBUG #INFO
logfile: /var/log/curator/curator.log
logformat: default
blacklist: ['elasticsearch', 'urllib3']
curator log output in DEBUG mode below:
2018-04-10 12:52:21,204 DEBUG curator.validators.filters f:48 Filter #1: {'filtertype': 'age', 'source': 'creation_date', 'direction': 'older', 'unit': 'days', 'unit_count': 30, 'timestring': None, 'stats_result': 'min_value', 'exclude': False, 'epoch': None}
2018-04-10 12:52:21,204 DEBUG curator.cli run:121 Full list of actions: {1: {'action': 'delete_indices', 'description': 'Delete indices older than 30 days (based on index name), for cloud_ prefixed indices. Ignore the error if the filter does not result in an actionable list of indices (ignore_empty_list) and exit cleanly.', 'options': {'ignore_empty_list': True, 'disable_action': False, 'continue_if_exception': False, 'timeout_override': None}, 'filters': [{'filtertype': 'pattern', 'kind': 'prefix', 'value': 'cloud_', 'exclude': False}, {'filtertype': 'age', 'source': 'creation_date', 'direction': 'older', 'unit': 'days', 'unit_count': 30, 'timestring': None, 'stats_result': 'min_value', 'exclude': False, 'epoch': None}]}}
2018-04-10 12:52:21,204 DEBUG curator.cli run:126 action_disabled = False
2018-04-10 12:52:21,204 DEBUG curator.cli run:130 continue_if_exception = False
2018-04-10 12:52:21,204 DEBUG curator.cli run:132 timeout_override = None
2018-04-10 12:52:21,204 DEBUG curator.cli run:134 ignore_empty_list = True
2018-04-10 12:52:21,204 INFO curator.cli run:144 Preparing Action ID: 1, "delete_indices"
2018-04-10 12:52:21,204 DEBUG curator.utils get_client:803 kwargs = {'hosts': [['_eth0_', '_local_']], 'port': 9200, 'use_ssl': True, 'ssl_no_validate': True, 'http_auth': 'elastic:<password>', 'master_only': False, 'aws_secret_key': None, 'aws_token': None, 'aws_sign_request': False, 'client_cert': None, 'client_key': None, 'aws_key': None, 'url_prefix': '', 'certificate': None, 'timeout': 60}
2018-04-10 12:52:21,207 DEBUG curator.utils get_client:880 Not using "requests_aws4auth" python module to connect.