Curator_cli show_snapshots doesn't seem to do anything or log output


(Simon) #1

I can't help but feel that I'm making a rookie mistake here but can't figure it out.

I'm trying to use curator_cli (version 5.5.3) to list snapshots on an ES cluster (version 6.2.4). It will happily show_indices and I can see the snapshots using curl. I've reduced it to the bare minimum (no config files any more) and still can't get it to work.

I have a repository called s3_backup. This is what happens when I use curl directly to list the snapshots:

$ curl -s http://localhost:9200/_snapshot/s3_backup/_all | jq .snapshots[].snapshot
"curator-20180510031502"
"curator-20180517031501"
"curator-20180518031543"
"curator-20180519031542"
"curator-20180520031542"
"curator-20180521031542"
"curator-20180522031542"
"curator-20180523031542"

When I try to do what I believe is the same thing using curator_cli I get nothing:

$ curator_cli --host 127.0.0.1 --port 9200 show_snapshots --repository s3_backup --filter_list '{"filtertype":"none"}'

If I do a show_indices this works as expected...

$ curator_cli --host 127.0.0.1 --port 9200 show_indices --filter_list '{"filtertype":"none"}'
.kibana
logstash-2018.05.16
logstash-2018.05.17
logstash-2018.05.18
logstash-2018.05.19
logstash-2018.05.20
logstash-2018.05.21
logstash-2018.05.22
logstash-2018.05.23

Adding --loglevel DEBUG makes this super noisy for show_indices:

curator_cli --loglevel DEBUG --host 127.0.0.1 --port 9200 show_indices --filter_list '{"filtertype":"none"}'
2018-05-23 12:43:23,475 DEBUG     curator.cli_singletons.cli_action.show_indices          check_filters:117  Validating provided filters: [{'filtertype': 'none'}]
2018-05-23 12:43:23,475 DEBUG     curator.validators.SchemaCheck               __init__:26   Schema: <function Filters.<locals>.f at 0x7f0477acfd90>
2018-05-23 12:43:23,475 DEBUG     curator.validators.SchemaCheck               __init__:27   "filters" config: [{'filtertype': 'none'}]
2018-05-23 12:43:23,475 DEBUG     curator.validators.SchemaCheck               __init__:26   Schema: {'filtertype': Any(In(['age', 'alias', 'allocated', 'closed', 'count', 'forcemerged', 'ilm', 'kibana', 'none', 'opened', 'pattern', 'period', 'space', 'state']), msg="filtertype must be one of ['age', 'alias', 'allocated', 'closed', 'count', 'forcemerged', 'ilm', 'kibana', 'none', 'opened', 'pattern', 'period', 'space', 'state']")}
2018-05-23 12:43:23,475 DEBUG     curator.validators.SchemaCheck               __init__:27   "filter" config: {'filtertype': 'none'}
2018-05-23 12:43:23,476 DEBUG     curator.validators.filters                      f:48   Filter #0: {'filtertype': 'none'}
2018-05-23 12:43:23,476 DEBUG              curator.utils             get_client:803  kwargs = {'hosts': ['127.0.0.1'], 'port': 9200, 'aws_sign_request': False, 'client_cert': None, 'timeout': 30, 'client_key': None, 'aws_secret_key': None, 'aws_key': None, 'master_only': False, 'http_auth': None, 'ssl_no_validate': False, 'url_prefix': '', 'aws_token': None, 'certificate': None, 'use_ssl': False}
2018-05-23 12:43:23,480 DEBUG              curator.utils             get_client:880  Not using "requests_aws4auth" python module to connect.
2018-05-23 12:43:23,484 DEBUG              curator.utils          check_version:689  Detected Elasticsearch version 6.2.4
2018-05-23 12:43:23,484 DEBUG          curator.indexlist          __get_indices:65   Getting all indices
2018-05-23 12:43:23,491 DEBUG              curator.utils            get_indices:644  Detected Elasticsearch version 6.2.4
...
<snip>

However if I add --loglevel DEBUG for show_snapshots it doesn't log any debug lines at all. I still get absolutely no output:

curator_cli --loglevel DEBUG --host 127.0.0.1 --port 9200 show_snapshots --repository s3_backup --filter_list '{"filtertype":"none"}'

strace shows that it is doing something, but doesn't suggest anything obvious.

strace curator_cli --loglevel DEBUG --host 127.0.0.1 --port 9200 show_snapshots --repository s3_backup --filter_list '{"filtertype":"none"}'
execve("/usr/bin/curator_cli", ["curator_cli", "--loglevel", "DEBUG", "--host", "127.0.0.1", "--port", "9200", "show_snapshots", "--repository", "s3_backup", "--filter_list", "{\"filtertype\":\"none\"}"], [/* 20 vars */]) = 0
brk(NULL)                               = 0x11f7000
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
readlink("/proc/self/exe", "/opt/elasticsearch-curator/curat"..., 4096) = 38
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/opt/elasticsearch-curator/tls/x86_64/libpthread.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/opt/elasticsearch-curator/tls/x86_64", 0x7fff9893b120) = -1 ENOENT (No such file or directory)
open("/opt/elasticsearch-curator/tls/libpthread.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/opt/elasticsearch-curator/tls", 0x7fff9893b120) = -1 ENOENT (No such file or directory)
open("/opt/elasticsearch-curator/x86_64/libpthread.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
...
<SNIP>
...
open("/opt/elasticsearch-curator/lib/library.zip", O_RDONLY) = 4
ioctl(4, FIOCLEX)                       = 0
fstat(4, {st_mode=S_IFREG|0664, st_size=912451, ...}) = 0
lseek(4, 0, SEEK_SET)                   = 0
read(4, "PK\3\4\24\0\0\0\10\0f\254\265L=.\nD\327\0\0\0\35\1\0\0\23\0\0\0BU"..., 2568) = 2568
read(4, "PK\3\4\24\0\0\0\10\0\256\243\265L\7\240\302\242s\2\0\0\332\3\0\0\17\0\0\0_b"..., 4096) = 4096
lseek(4, 6664, SEEK_SET)                = 6664
close(4)                                = 0
stat("/root/.curator/curator.yml", 0x7fff989387d0) = -1 ENOENT (No such file or directory)
stat("/root/.curator/curator.yml", 0x7fff989395d0) = -1 ENOENT (No such file or directory)
rt_sigaction(SIGINT, {SIG_DFL, [], SA_RESTORER, 0x7f4ef7768390}, {0x5761f0, [], SA_RESTORER, 0x7f4ef7768390}, 8) = 0
munmap(0x7f4ef2987000, 262144)          = 0
sigaltstack(NULL, {ss_sp=0x1215400, ss_flags=0, ss_size=8192}) = 0
sigaltstack({ss_sp=NULL, ss_flags=SS_DISABLE, ss_size=0}, NULL) = 0
close(3)                                = 0
exit_group(0)                           = ?
+++ exited with 0 +++

If anyone can shed any light on what I'm doing wrong that would be really helpful. In the meanwhile I'm going to prototype using curl as I feel like I've reached the end of a confusing road. :confounded:

Thanks for reading!
Simon


(Aaron Mildenstein) #2

It means I missed something. Watch for 5.5.4 to be released soon.


(Aaron Mildenstein) #3

When the CI tests show the build is complete, I will do a quick release of 5.5.4.


(Simon) #4

Thanks @theuntergeek!

Out of interest how would I have troubleshot this further?


(Aaron Mildenstein) #5

I don't think you could have without running from source out of the repository.

$ git clone https://github.com/elastic/curator/curator.git
$ git checkout 5.x
$ python run_singleton.py show_snapshots --repository XXXXX

I added tracing to the show_snapshots method in curator/cli_singletons/show.py, and then realized that I had left extra args in the method that were not being populated, so it was failing. All I had to do was remove those, and it ran.

I'm embarrassed, because I already fixed those once on a test platform, but didn't merge the code in properly, so it didn't make it into 5.5.3.


(Simon) #6

No worries @theuntergeek. It happens to the best of us.

Many thanks for the quick replies.


(Aaron Mildenstein) #7

Curator 5.5.4 is released. The documentation site won't have the windows download links updated for at least 30 minutes (our doc builds are automated), but the RPM/DEB repos have 5.5.4, as does PyPi.


(system) #8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.