# Unable to configure curator to archive data from Elasticsearch
**URL:** https://discuss.elastic.co/t/unable-to-configure-curator-to-archive-data-from-elasticsearch/323354
**Category:** Elasticsearch
**Tags:** curator
**Created:** [January 17, 2023, 5:56pm UTC](https://discuss.elastic.co/t/unable-to-configure-curator-to-archive-data-from-elasticsearch/323354 "2023-01-17T17:56:58Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@Pendela-BhargavaSai](https://discuss.elastic.co/u/Pendela-BhargavaSai)
#### Post date: [January 17, 2023, 5:56pm UTC](https://discuss.elastic.co/t/unable-to-configure-curator-to-archive-data-from-elasticsearch/323354/1 "2023-01-17T17:56:58Z")
Hi I am a newbie to Elastic search.
In my project we are working on archiving data using Curator. I am trying to configure the curator with Elasticsearch but unable to do that. I am facing some connectivity issues for connecting my project Elasticsearch.
My configuration yaml file
My action yaml file
When I am running this command ( curator --config /home/ec2-user/curator/config.yml /home/ec2-user/curator/action.yml) in EC2 instance, I am getting a log report like below
I already configured my AWS creds in the .aws/credentials path using gimme-aws-cred plugin on EC2. And my curator version is 5.8.4 and my Elasticsearch version is 6.8.0.
Anyone Please help me know if I am missing anything in the process or do I need to add anything to this process?
---
### Author:  [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)
#### Post date: [January 17, 2023, 6:53pm UTC](https://discuss.elastic.co/t/unable-to-configure-curator-to-archive-data-from-elasticsearch/323354/2 "2023-01-17T18:53:58Z")
The `type is missing` message may indicate that you have a much more recent `elasticsearch-py` python module since the code indicates it can use anything [`elasticsearch>=7.12.0,<8.0`](https://github.com/elastic/curator/blob/v5.8.4/requirements.txt#LL2-L2C29), and that sounds like an API change. It _should_ work with 6.8.0 still with the 7.12 client version, but if not, you may need to use an older version of Curator that does work with 6.8.0 properly.
In any case, setting `loglevel: DEBUG` is a better way to see what is going on behind the scenes, and sending pre-formatted text is always preferred to screenshots for sharing here.
---
### Author:  [@Pendela-BhargavaSai](https://discuss.elastic.co/u/Pendela-BhargavaSai)
#### Post date: [January 20, 2023, 7:44am UTC](https://discuss.elastic.co/t/unable-to-configure-curator-to-archive-data-from-elasticsearch/323354/3 "2023-01-20T07:44:30Z")
Hi Aaron,
I downgraded my curator version to 5.4.1 and ran the same config and action file but got the same error messages.
**Command I ran in EC2:-**
curator --config /home/ec2-user/curator/config.yml /home/ec2-user/curator/action.yml
**Error message:-**
/home/ec2-user/.local/lib/python3.7/site-packages/curator/utils.py:53: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read [https://msg.pyyaml.org/load](https://msg.pyyaml.org/load) for full details.
cfg = yaml.load(raw)
Traceback (most recent call last):
File "/home/ec2-user/.local/lib/python3.7/site-packages/curator/utils.py", line 896, in get\_client
check\_version(client)
File "/home/ec2-user/.local/lib/python3.7/site-packages/curator/utils.py", line 686, in check\_version
version\_number = get\_version(client)
File "/home/ec2-user/.local/lib/python3.7/site-packages/curator/utils.py", line 659, in get\_version
version = client.info()['version']['number']
File "/home/ec2-user/.local/lib/python3.7/site-packages/elasticsearch/client/utils.py", line 101, in \_wrapped
return func(\*args, params=params, \*\*kwargs)
File "/home/ec2-user/.local/lib/python3.7/site-packages/elasticsearch/client/ **init**.py", line 265, in info
return self.transport.perform\_request("GET", "/", params=params)
File "/home/ec2-user/.local/lib/python3.7/site-packages/elasticsearch/transport.py", line 409, in perform\_request
timeout=timeout,
File "/home/ec2-user/.local/lib/python3.7/site-packages/elasticsearch/connection/http\_requests.py", line 186, in perform\_request
self.\_raise\_error(response.status\_code, raw\_data)
File "/home/ec2-user/.local/lib/python3.7/site-packages/elasticsearch/connection/base.py", line 254, in \_raise\_error
status\_code, error\_message, additional\_info
elasticsearch.exceptions.RequestError: RequestError(400, 'action\_request\_validation\_exception', 'Validation Failed: 1: type is missing;')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/bin/curator", line 8, in
sys.exit(cli())
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 829, in **call**
return self.main(\*args, \*\*kwargs)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, \*\*ctx.params)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 610, in invoke
return callback(\*args, \*\*kwargs)
File "/home/ec2-user/.local/lib/python3.7/site-packages/curator/cli.py", line 211, in cli
run(config, action\_file, dry\_run)
File "/home/ec2-user/.local/lib/python3.7/site-packages/curator/cli.py", line 158, in run
client = get\_client(\*\*client\_args)
File "/home/ec2-user/.local/lib/python3.7/site-packages/curator/utils.py", line 903, in get\_client
'Error: {0}'.format(e)
elasticsearch.exceptions.ElasticsearchException: Unable to create client connection to Elasticsearch. Error: RequestError(400, 'action\_request\_validation\_exception', 'Validation Failed: 1: type is missing;')
---
### Author:  [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)
#### Post date: [January 20, 2023, 5:49pm UTC](https://discuss.elastic.co/t/unable-to-configure-curator-to-archive-data-from-elasticsearch/323354/4 "2023-01-20T17:49:20Z")
I am no longer able to troubleshoot these older versions as directly. I can see that you downgraded to 5.4.1, which is older than it needs to be, I think, for even 6.8.0.
You should not have anything set for `url_prefix`. It is not needed here.
it sounds like there may be other issues with this installation. Please install the RPM or DEB based package for Curator 5.8.4 if you can. It bundles the appropriate python modules with it, and it seems that you have some conflicts, still.
---
### Author:  [@Pendela-BhargavaSai](https://discuss.elastic.co/u/Pendela-BhargavaSai)
#### Post date: [January 28, 2023, 10:08am UTC](https://discuss.elastic.co/t/unable-to-configure-curator-to-archive-data-from-elasticsearch/323354/5 "2023-01-28T10:08:18Z")
Hi Aaron,
I got a solution for my problem and Curator is working fine. I have updated all my plugins and dependencies on my working instance and also I used " **sudo amazon-linux-extras install epel -y**" this command for adding some more dependency plugins if needed. Now I am able to archive the old data. Thanks for the Guidance😊.
But I have a query related to delete index. When I am trying to delete index it is going to delete complete index based on given index name or timestamp condition with directions. But I want to delete only particular data in-between certain dates or given dates(condition).
Is there any possibility or a-way to do that using Curator?
Or delete-by-query is the only option we need to use?
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 25, 2023, 10:09am UTC](https://discuss.elastic.co/t/unable-to-configure-curator-to-archive-data-from-elasticsearch/323354/6 "2023-02-25T10:09:12Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
---
### Author:  [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)
#### Post date: [March 20, 2023, 3:37pm UTC](https://discuss.elastic.co/t/unable-to-configure-curator-to-archive-data-from-elasticsearch/323354/7 "2023-03-20T15:37:39Z")
Curator only works at the whole index level. Anything intra index is `delete_by_query` only.