Docker Curator is not working

Elastic Stack Elasticsearch
1

Hello,

I got the exact same issue of this post :

Which is :

Traceback (most recent call last):
File "/usr/bin/curator", line 11, in
load_entry_point('elasticsearch-curator==5.7.3', 'console_scripts', 'curator')()
File "/usr/lib/python2.7/site-packages/click/core.py", line 722, in call
return self.main(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/lib/python2.7/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/lib/python2.7/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/curator/cli.py", line 218, in cli
run(config, action_file, dry_run)
File "/usr/lib/python2.7/site-packages/curator/cli.py", line 165, in run
client = get_client(**client_args)
File "/usr/lib/python2.7/site-packages/curator/utils.py", line 906, in get_client
'Error: {0}'.format(e)
elasticsearch.exceptions.ElasticsearchException: Unable to create client connection to Elasticsearch. Error: Unknown mimetype, unable to deserialize: text/html

I installed elk with docker and want to add the curator extension, but it cannot reach Elasticsearch.

Do someone have a clue to solve this issue ?

2

Whose docker image are you using? I highly recommend using something based on Python 3.6. Python 2.7 is literally about to be officially EOL, and there are quite a few outdated libraries which could contribute to this in non-updated, non-recent versions of Python 2.7. I recommend at least Python 2.7.10 if you are using that version. Even then, it's hit or miss.

The official package builds of Curator (RPM/DEB/Windows) are all based on Python 3.6, as it is much more up-to-date, and has many improved SSL library dependencies out of the box.

3

Hi !

Thanks for your help.

I will try to upgrade Python's version.

Here's my Dockerfile in case you need to see it :

FROM alpine:3.8

ENV CURATOR_VERSION=5.7.3

RUN apk --update add --no-cache tini python py-pip
&& pip install elasticsearch-curator==${CURATOR_VERSION}

COPY entrypoint.sh /
RUN chmod +x /entrypoint.sh

WORKDIR /usr/share/curator
COPY config ./config

ENTRYPOINT ["/entrypoint.sh"]

4

I highly recommend using the Dockerfile from Curator's GitHub repository as the basis of your Dockerfile. It gets certificates and other niceties for your container.

1 Like
5

Hi,

First, thank you so much for your reply.
I installed the curator from the gitHub repository you had mentioned.

But I do not see how to start it with cron and parameters in the readme file.
Plus, I got the following error in my container logs :

Usage: curator [OPTIONS] ACTION_FILE

Error: Missing argument "action_file".

6

This means Curator expects you to provide optional [OPTIONS] and an ACTION_FILE.

If your cluster is not running on port 9200 of the local host, one of the necessary options is a configuration file, passed to the command-line as --config /path/to/FILENAME.YML.

As for cron, here's a tutorial.

7

I'm sorry, I know I'm missing something but I still not see how to give action_file.yml and curator.yml to the container.

I searched where the "curator --config curator.yml action_file" is launch in the GitHub project but I didn't found it.

I tried to launch docker-compose command with some environment variables like ACTION_FILE but it still not work.

For the curator.yml file I have the solution to create the default ~/.curator/curator.yml.
But there is no default file for action_file.

8

You need to add the mapping for those on your own. Or you can use curator_cli for the entry and provide all of the values as flags, which is a common option.

9

Ok I get it.
I keept the command entrypoint of github and add my own action_file.yml.

ENTRYPOINT ["/curator/curator", "/usr/share/curator/config/action_file.yml"]

But I'm back on the error I had first:

2019-07-23 09:26:21,943 INFO curator.utils get_client:907 Testing client connectivity
2019-07-23 09:26:21,971 ERROR curator.utils get_client:911 Unable to connect to Elasticsearch cluster. Error: Unknown mimetype, unable to deserialize: text/html
2019-07-23 09:26:21,971 CRITICAL curator.utils get_client:912 Curator cannot proceed. Exiting.

Weird because I'm using the same network than elasticsearch...

10

I am debugging the python file. Something is strange:
In get_client method at the line 809 kwargs is containing the good configuration (curator.yml).
But when I log the "client" object, it seems to use port 443 instead of 9210.

2019-07-23 12:15:42,996 DEBUG curator.utils get_client:809 kwargs = {'hosts': ['https://elk-tst.tmhmf.net'], 'port': 9210, 'use_ssl': False, 'ssl_no_validate': False, 'master_only': False, 'url_prefix': '', 'aws_secret_key': None, 'client_cert': None, 'aws_token': None, 'http_auth': None, 'aws_sign_request': False, 'certificate': None, 'client_key': None, 'api_key': None, 'aws_key': None, 'timeout': 30}
2019-07-23 12:15:42,996 DEBUG curator.utils get_client:869 Checking for AWS settings
2019-07-23 12:15:43,000 DEBUG curator.utils get_client:887 Not using "requests_aws4auth" python module to connect.
2019-07-23 12:15:43,000 INFO curator.utils get_client:900 Instantiating client object
2019-07-23 12:15:43,001 INFO curator.utils get_client:902 Created Elasticsearch client object with provided settings
2019-07-23 12:15:43,001 INFO curator.utils get_client:907 Testing client connectivity
2019-07-23 12:15:43,001 DEBUG curator.utils get_client:908 TEMP TEST !!! <Elasticsearch([{'host': 'elk-tst.tmhmf.net', 'port': 443, 'use_ssl': True}])>
2019-07-23 12:15:43,029 ERROR curator.utils get_client:912 Unable to connect to Elasticsearch cluster. Error: Unknown mimetype, unable to deserialize: text/html

11

You've specified a URL, not a hostname, which you can do, but it overrides the port if you do so. This is why it changed to port 443 (you specified https in the URL). You should only have an IP or a fully-qualified domain name in the hosts array if you want to use port. Or you can keep the URL and include the port at the end of the URL. You should probably also specify use_ssl: true if you are connecting to an SSL endpoint.

12

It finally works !!!

Thank you so much !

13

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.