Unable to use curator 5.2.0 with ELK 5.6.2

Elastic Stack Elasticsearch
1

Hi,

I have ELK 5.6.0 installed and I want to delete data older than 10 days. I have installed curator using pip and it got ver 5.2.0 of it by default.

When I execute curator I get following error

[root@elk0 elasticsearch-curator]# /usr/bin/curator –config /elk/sw/elasticsearch-curator/config.yml –dry-run /elk/sw/elasticsearch-curator/action.yml
Traceback (most recent call last):
File "/usr/bin/curator", line 5, in
from pkg_resources import load_entry_point
File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 2655, in
working_set.require(requires)
File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 648, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 546, in resolve
raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: certifi>=2017.4.17

I am on RHEL 9 and I do have certifi (2017.7.27.1) installed.

pls help

Regards,
-Manish

2

I presume you installed via pip, as this appears to be using the system version of Python 2.6. Python 2.6 is not officially supported by Curator anymore, as it is fully deprecated/not supported by the Python organization anymore.

Is there a reason you chose not to install from the YUM repository? This version is self-contained with all the necessary Python 3.6 modules and system libraries, and should work independently of the built-in Python on RHEL/CentOS systems.

3

Hi Aaron, I had first installed it using YUM repo but upon executing curator I got GLIBC errors. then I tried pip installation.

Regards,
-Manish

4

Hi,

I have now installed python 3 from source and have installed curator with pip3.

Now I am not getting any system error but I see usage error. I am able to use same arguments in curator 5.1.1 but it 5.2 it gives error

[root@elk0 curator]# /elk/sw/python3.6.3/bin/curator –-config /elk/sw/elasticsearch-curator/config.yml --dry-run /elk/sw/elasticsearch-curator/action.yml
Usage: curator [OPTIONS] ACTION_FILE

Error: Invalid value for "action_file": Path "–-config" does not exist.

Regards,
-Manish

5

Try putting --dry-run before --config

6

Also, looking closer, and editing your pasted response to be encapsulated in triple back-ticks, your version doesn't say --config, but rather –-config. I'm not sure what character that first dash is, but it is not the regular -, and that's why it's being misinterpreted. The dashes are clearly correct for your --dry-run flag, though.

7

Hi Aaron,

You were correct, when I tried using --dry-run before --config it took the cmd. But now I am getting different error.

my config.yml (curator config) is:

client:
hosts:
- 127.0.0.1
port: 9200
logging:
loglevel: INFO
logfile: "/var/log/curator/actions.log"
logformat: default
blacklist: ['elasticsearch', 'urllib3']

And my action.yml is:

actions:
1:
action: delete_indices
description: >-
Delete indices older than 10 days (based on index name), for logstash-
prefixed indices. Ignore the error if the filter does not result in an
actionable list of indices (ignore_empty_list) and exit cleanly.
options:
ignore_empty_list: True
timeout_override:
continue_if_exception: False
disable_action: False
filters:
- filtertype: pattern
kind: prefix
value: logstash-
exclude:
- filtertype: age
source: name
direction: older
timestring: '%Y.%m.%d'
unit: days
unit_count: 10
exclude:

When I execute curator I get following error.

[root@elk0 bin]# /elk/sw/python3.6.3/bin/curator --dry-run --config /elk/sw/elasticsearch-curator/config.yml /elk/sw/elasticsearch-curator/action.yml

Schema error: extra keys not allowed @ data['hosts']
Traceback (most recent call last):
_ File "/elk/sw/python3.6.3/lib/python3.6/site-packages/curator/validators/schemacheck.py", line 57, in result_
_ return self.schema(self.config)_
_ File "/elk/sw/python3.6.3/lib/python3.6/site-packages/voluptuous/schema_builder.py", line 221, in call_
_ return self.compiled([], data)
_ File "/elk/sw/python3.6.3/lib/python3.6/site-packages/voluptuous/schema_builder.py", line 538, in validate_dict_
_ return base_validate(path, iteritems(data), out)_
_ File "/elk/sw/python3.6.3/lib/python3.6/site-packages/voluptuous/schema_builder.py", line 370, in validate_mapping_
_ raise er.MultipleInvalid(errors)_
voluptuous.error.MultipleInvalid: extra keys not allowed @ data['hosts']

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
_ File "/elk/sw/python3.6.3/bin/curator", line 11, in _
_ load_entry_point('elasticsearch-curator==5.2.0', 'console_scripts', 'curator')()_
_ File "/elk/sw/python3.6.3/lib/python3.6/site-packages/click/core.py", line 722, in call_
_ return self.main(*args, **kwargs)_
_ File "/elk/sw/python3.6.3/lib/python3.6/site-packages/click/core.py", line 697, in main_
_ rv = self.invoke(ctx)_
_ File "/elk/sw/python3.6.3/lib/python3.6/site-packages/click/core.py", line 895, in invoke_
_ return ctx.invoke(self.callback, **ctx.params)_
_ File "/elk/sw/python3.6.3/lib/python3.6/site-packages/click/core.py", line 535, in invoke_
_ return callback(*args, **kwargs)_
_ File "/elk/sw/python3.6.3/lib/python3.6/site-packages/curator/cli.py", line 211, in cli_
_ run(config, action_file, dry_run)_
_ File "/elk/sw/python3.6.3/lib/python3.6/site-packages/curator/cli.py", line 106, in run_
_ client_args = process_config(config)_
_ File "/elk/sw/python3.6.3/lib/python3.6/site-packages/curator/config_utils.py", line 45, in process_config_
_ config = test_config(yaml_file)_
_ File "/elk/sw/python3.6.3/lib/python3.6/site-packages/curator/config_utils.py", line 19, in test_config_
_ 'Client Configuration', 'full configuration dictionary').result()_
_ File "/elk/sw/python3.6.3/lib/python3.6/site-packages/curator/validators/schemacheck.py", line 68, in result_
_ self.test_what, self.location, self.badvalue, self.error)_
curator.exceptions.ConfigurationError: Configuration: Client Configuration: Location: full configuration dictionary: Bad Value: "['127.0.0.1']", extra keys not allowed @ data['hosts']. Check configuration file.

Any idea how to fix this ? same action and config works well in curator 5.1.1

Regards,
-Manish

8

This indicates something was not properly indented. YAML requires proper indentation. You need to paste your output and configuration inside code tags, or triple back-ticks:

```
Pasted text
```

This will format the text as a mono space font, like in a console. It shows indentation properly.

9 
**Corrected action.yml** 

[root@elk0 marya]# cat /elk/sw/elasticsearch-curator/action.yml
---
actions:
  1:
    action: delete_indices
    description: >-
      Delete indices older than 10 days (based on index name), for logstash-
      prefixed indices. Ignore the error if the filter does not result in an
      actionable list of indices (ignore_empty_list) and exit cleanly.
    options:
      ignore_empty_list: True
      timeout_override:
      continue_if_exception: False
      disable_action: False
    filters:
    - filtertype: pattern
      kind: prefix
      value: logstash-
      exclude:
    - filtertype: age
      source: name
      direction: older
      timestring: '%Y.%m.%d'
      unit: days
      unit_count: 90
      exclude:

**Config.yml**

[root@elk0 marya]# cat /elk/sw/elasticsearch-curator/config.yml
---
client:
  hosts:
   - 127.0.0.1
  port: 9200
logging:
  loglevel: INFO
  logfile: "/var/log/curator/actions.log"
  logformat: default
  blacklist: ['elasticsearch', 'urllib3']
10

This should probably be a single line, unless you plan on adding more hosts in the future:

---
client:
  hosts: 127.0.0.1

or

---
client:
  hosts: [ 127.0.0.1 ]
11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.