Curator_cli and curator commands do nothing

I have a pair of ES 6.1.1 clusters and have deployed curator 5.4.1 against one of them. Curator and curator_cli commands have no effect. Logging is relatively sparse, however in debug mode I get some output, which I've included along with config files below.

I've disabled ssl validation as I'm using self-signed certs for this instance. I see no errors suggesting this is an issue, but thought it worthy of note for troubleshooting purposes.

Do you have any suggestions for things to explore in troubleshooting this issue?

Thanks.

/etc/elasticsearch/elasticsearch.yml below:

---
cluster:
  name: elk_test_cluster
  routing:
    allocation:
      allow_rebalance: always
      disk:
        watermark:
          low: 90%
          high: 90%
      awareness:
        attributes: aws_availability_zone
        force:
          aws_availability_zone:
            values: us-east-1a,us-east-1b
path:
  data: "/mnt/elasticsearch"
  logs: "/var/log/elasticsearch/"
network:
  host:
  - _eth0_ # _ec2_
  - _local_
  bind_host:
  - _eth0_ # _ec2_
  - _local_
  publish_host:
  - _eth0_ # _ec2_
  - _local_
http:
  port: '9200'
xpack:
  security:
    enabled: true
    audit:
      enabled: true
    http:
      ssl:
        enabled: true
    transport:
      ssl:
        enabled: true
        verification_mode: certificate
    authc:
      realms:
        realm1:
          type: native
          order: '0'
          enabled: true
        realm2:
          type: file
          order: '1'
          enabled: true
  monitoring:
    enabled: true
  graph:
    enabled: true
  watcher:
    enabled: true
  ssl:
    key: "/etc/elasticsearch/x-pack/elasticsearch.key"
    certificate: "/etc/elasticsearch/x-pack/elasticsearch.crt"
    certificate_authorities:
    - "/etc/elasticsearch/x-pack/ca.crt"
  notification:
    email:
      account:
        smtp_account:
          profile: standard
          smtp:
            host: localhost
            from: nobody@nowhere.com
cloud:
  node:
    auto_attributes: 'true'
discovery:
  zen:
    hosts_provider: ec2
  ec2:
    tag:
      efk_cluster_name: elk_test_cluster
    host_type: private_ip
    availability_zones:
    - us-east-1a
    - us-east-1b
    - us-east-1c
    - us-east-1d
    - us-east-1e
    endpoint: ec2.us-east-1.amazonaws.com
node:
  name: ip-XXX-XX-XX-XXX
  master: 'true'
  data: 'false'
  ingest: 'false'
thread_pool:
  bulk:
    queue_size: '1000'
  search:
    queue_size: '10000'

/etc/curator/config.yml below:

# Remember, leave a key empty if there is no value.  None will be a string,
# not a Python "NoneType"
client:
  hosts:
    - ["_eth0_", "_local_"]
  port: 9200
  url_prefix:
  use_ssl: True
  certificate:
  client_cert:
  client_key:
  ssl_no_validate: True
  http_auth: elastic:<PASSWORD>
  timeout: 60
  master_only: False
logging:
  loglevel: DEBUG #INFO
  logfile: /var/log/curator/curator.log
  logformat: default
  blacklist: ['elasticsearch', 'urllib3']

curator log output in DEBUG mode below:

2018-04-10 12:52:21,204 DEBUG     curator.validators.filters                      f:48   Filter #1: {'filtertype': 'age', 'source': 'creation_date', 'direction': 'older', 'unit': 'days', 'unit_count': 30, 'timestring': None, 'stats_result': 'min_value', 'exclude': False, 'epoch': None}
2018-04-10 12:52:21,204 DEBUG                curator.cli                    run:121  Full list of actions: {1: {'action': 'delete_indices', 'description': 'Delete indices older than 30 days (based on index name), for cloud_ prefixed indices. Ignore the error if the filter does not result in an actionable list of indices (ignore_empty_list) and exit cleanly.', 'options': {'ignore_empty_list': True, 'disable_action': False, 'continue_if_exception': False, 'timeout_override': None}, 'filters': [{'filtertype': 'pattern', 'kind': 'prefix', 'value': 'cloud_', 'exclude': False}, {'filtertype': 'age', 'source': 'creation_date', 'direction': 'older', 'unit': 'days', 'unit_count': 30, 'timestring': None, 'stats_result': 'min_value', 'exclude': False, 'epoch': None}]}}
2018-04-10 12:52:21,204 DEBUG                curator.cli                    run:126  action_disabled = False
2018-04-10 12:52:21,204 DEBUG                curator.cli                    run:130  continue_if_exception = False
2018-04-10 12:52:21,204 DEBUG                curator.cli                    run:132  timeout_override = None
2018-04-10 12:52:21,204 DEBUG                curator.cli                    run:134  ignore_empty_list = True
2018-04-10 12:52:21,204 INFO                 curator.cli                    run:144  Preparing Action ID: 1, "delete_indices"
2018-04-10 12:52:21,204 DEBUG              curator.utils             get_client:803  kwargs = {'hosts': [['_eth0_', '_local_']], 'port': 9200, 'use_ssl': True, 'ssl_no_validate': True, 'http_auth': 'elastic:<password>', 'master_only': False, 'aws_secret_key': None, 'aws_token': None, 'aws_sign_request': False, 'client_cert': None, 'client_key': None, 'aws_key': None, 'url_prefix': '', 'certificate': None, 'timeout': 60}
2018-04-10 12:52:21,207 DEBUG              curator.utils             get_client:880  Not using "requests_aws4auth" python module to connect.

This usage is not supported. Curator uses urllib3 to connect to a REST endpoint, and does not recognize these elasticsearch.yml-specific identifiers. You should specify a single IP address or host name per line, e.g.:

client:
  hosts:
    - 127.0.0.1
    - 10.0.0.1
...

See more in the hosts documentation.

Thanks @theuntergeek I'll give that change a try and look over the docs.
I inherited this cluster and am learning as I go, so this feedback is helpful, thank you.

I made that update and haven't seen any changes. Do you have any other suggestions?

# Remember, leave a key empty if there is no value.  None will be a string,
# not a Python "NoneType"
client:
  hosts:
  #  - ["_eth0_", "_local_"]
    - ["127.0.0.1"]
  port: 9200
  url_prefix:
  use_ssl: True
  certificate:
  client_cert:
  client_key:
  ssl_no_validate: True
  http_auth: elastic:<password>
  timeout: 60
  master_only: False
logging:
  loglevel: DEBUG #INFO
  logfile: /var/log/curator/curator.log
  logformat: default
  blacklist: ['elasticsearch', 'urllib3']
[user@ip-XXX-XXX-XXX-XXX ~]# vim /etc/curator/config.yml
[user@ip-XXX-XXX-XXX-XXX ~]# !curator_cli
curator_cli --config /etc/curator/config.yml show_indices --filter_list '[{"filtertype":"pattern","kind":"regex","value":"indexname-2018-03-31"}]'

log:

2018-04-11 08:53:42,468 DEBUG              curator.utils             get_client:803  kwargs = {'hosts': [['127.0.0.1']], 'port': 9200, 'use_ssl': True, 'ssl_no_validate': True, 'http_auth': 'elastic:<password>', 'timeout': 60, 'master_only': False, 'url_prefix': '', 'aws_sign_request': False, 'aws_token': None, 'aws_key': None, 'aws_secret_key': None, 'certificate': None, 'client_cert': None, 'client_key': None}
2018-04-11 08:53:42,471 DEBUG              curator.utils             get_client:880  Not using "requests_aws4auth" python module to connect.

You added square brackets to YAML array notification, making the host entry a nested array. If you take another look at what I put forward, it looked like:

Note the absence of square braces. The subject of hosts configuration and YAML array notation is covered extensively in the hosts documentation.

1 Like

Thanks.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.