Curator_cli show_snapshots doesn't seem to do anything or log output

I can't help but feel that I'm making a rookie mistake here but can't figure it out.

I'm trying to use curator_cli (version 5.5.3) to list snapshots on an ES cluster (version 6.2.4). It will happily show_indices and I can see the snapshots using curl. I've reduced it to the bare minimum (no config files any more) and still can't get it to work.

I have a repository called s3_backup. This is what happens when I use curl directly to list the snapshots:

$ curl -s http://localhost:9200/_snapshot/s3_backup/_all | jq .snapshots[].snapshot
"curator-20180510031502"
"curator-20180517031501"
"curator-20180518031543"
"curator-20180519031542"
"curator-20180520031542"
"curator-20180521031542"
"curator-20180522031542"
"curator-20180523031542"

When I try to do what I believe is the same thing using curator_cli I get nothing:

$ curator_cli --host 127.0.0.1 --port 9200 show_snapshots --repository s3_backup --filter_list '{"filtertype":"none"}'

If I do a show_indices this works as expected...

$ curator_cli --host 127.0.0.1 --port 9200 show_indices --filter_list '{"filtertype":"none"}'
.kibana
logstash-2018.05.16
logstash-2018.05.17
logstash-2018.05.18
logstash-2018.05.19
logstash-2018.05.20
logstash-2018.05.21
logstash-2018.05.22
logstash-2018.05.23

Adding --loglevel DEBUG makes this super noisy for show_indices:

curator_cli --loglevel DEBUG --host 127.0.0.1 --port 9200 show_indices --filter_list '{"filtertype":"none"}'
2018-05-23 12:43:23,475 DEBUG     curator.cli_singletons.cli_action.show_indices          check_filters:117  Validating provided filters: [{'filtertype': 'none'}]
2018-05-23 12:43:23,475 DEBUG     curator.validators.SchemaCheck               __init__:26   Schema: <function Filters.<locals>.f at 0x7f0477acfd90>
2018-05-23 12:43:23,475 DEBUG     curator.validators.SchemaCheck               __init__:27   "filters" config: [{'filtertype': 'none'}]
2018-05-23 12:43:23,475 DEBUG     curator.validators.SchemaCheck               __init__:26   Schema: {'filtertype': Any(In(['age', 'alias', 'allocated', 'closed', 'count', 'forcemerged', 'ilm', 'kibana', 'none', 'opened', 'pattern', 'period', 'space', 'state']), msg="filtertype must be one of ['age', 'alias', 'allocated', 'closed', 'count', 'forcemerged', 'ilm', 'kibana', 'none', 'opened', 'pattern', 'period', 'space', 'state']")}
2018-05-23 12:43:23,475 DEBUG     curator.validators.SchemaCheck               __init__:27   "filter" config: {'filtertype': 'none'}
2018-05-23 12:43:23,476 DEBUG     curator.validators.filters                      f:48   Filter #0: {'filtertype': 'none'}
2018-05-23 12:43:23,476 DEBUG              curator.utils             get_client:803  kwargs = {'hosts': ['127.0.0.1'], 'port': 9200, 'aws_sign_request': False, 'client_cert': None, 'timeout': 30, 'client_key': None, 'aws_secret_key': None, 'aws_key': None, 'master_only': False, 'http_auth': None, 'ssl_no_validate': False, 'url_prefix': '', 'aws_token': None, 'certificate': None, 'use_ssl': False}
2018-05-23 12:43:23,480 DEBUG              curator.utils             get_client:880  Not using "requests_aws4auth" python module to connect.
2018-05-23 12:43:23,484 DEBUG              curator.utils          check_version:689  Detected Elasticsearch version 6.2.4
2018-05-23 12:43:23,484 DEBUG          curator.indexlist          __get_indices:65   Getting all indices
2018-05-23 12:43:23,491 DEBUG              curator.utils            get_indices:644  Detected Elasticsearch version 6.2.4
...
<snip>

However if I add --loglevel DEBUG for show_snapshots it doesn't log any debug lines at all. I still get absolutely no output:

curator_cli --loglevel DEBUG --host 127.0.0.1 --port 9200 show_snapshots --repository s3_backup --filter_list '{"filtertype":"none"}'

strace shows that it is doing something, but doesn't suggest anything obvious.

strace curator_cli --loglevel DEBUG --host 127.0.0.1 --port 9200 show_snapshots --repository s3_backup --filter_list '{"filtertype":"none"}'
execve("/usr/bin/curator_cli", ["curator_cli", "--loglevel", "DEBUG", "--host", "127.0.0.1", "--port", "9200", "show_snapshots", "--repository", "s3_backup", "--filter_list", "{\"filtertype\":\"none\"}"], [/* 20 vars */]) = 0
brk(NULL)                               = 0x11f7000
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
readlink("/proc/self/exe", "/opt/elasticsearch-curator/curat"..., 4096) = 38
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/opt/elasticsearch-curator/tls/x86_64/libpthread.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/opt/elasticsearch-curator/tls/x86_64", 0x7fff9893b120) = -1 ENOENT (No such file or directory)
open("/opt/elasticsearch-curator/tls/libpthread.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/opt/elasticsearch-curator/tls", 0x7fff9893b120) = -1 ENOENT (No such file or directory)
open("/opt/elasticsearch-curator/x86_64/libpthread.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
...
<SNIP>
...
open("/opt/elasticsearch-curator/lib/library.zip", O_RDONLY) = 4
ioctl(4, FIOCLEX)                       = 0
fstat(4, {st_mode=S_IFREG|0664, st_size=912451, ...}) = 0
lseek(4, 0, SEEK_SET)                   = 0
read(4, "PK\3\4\24\0\0\0\10\0f\254\265L=.\nD\327\0\0\0\35\1\0\0\23\0\0\0BU"..., 2568) = 2568
read(4, "PK\3\4\24\0\0\0\10\0\256\243\265L\7\240\302\242s\2\0\0\332\3\0\0\17\0\0\0_b"..., 4096) = 4096
lseek(4, 6664, SEEK_SET)                = 6664
close(4)                                = 0
stat("/root/.curator/curator.yml", 0x7fff989387d0) = -1 ENOENT (No such file or directory)
stat("/root/.curator/curator.yml", 0x7fff989395d0) = -1 ENOENT (No such file or directory)
rt_sigaction(SIGINT, {SIG_DFL, [], SA_RESTORER, 0x7f4ef7768390}, {0x5761f0, [], SA_RESTORER, 0x7f4ef7768390}, 8) = 0
munmap(0x7f4ef2987000, 262144)          = 0
sigaltstack(NULL, {ss_sp=0x1215400, ss_flags=0, ss_size=8192}) = 0
sigaltstack({ss_sp=NULL, ss_flags=SS_DISABLE, ss_size=0}, NULL) = 0
close(3)                                = 0
exit_group(0)                           = ?
+++ exited with 0 +++

If anyone can shed any light on what I'm doing wrong that would be really helpful. In the meanwhile I'm going to prototype using curl as I feel like I've reached the end of a confusing road. :confounded:

Thanks for reading!
Simon

It means I missed something. Watch for 5.5.4 to be released soon.

When the CI tests show the build is complete, I will do a quick release of 5.5.4.

Thanks @theuntergeek!

Out of interest how would I have troubleshot this further?

I don't think you could have without running from source out of the repository.

$ git clone https://github.com/elastic/curator/curator.git
$ git checkout 5.x
$ python run_singleton.py show_snapshots --repository XXXXX

I added tracing to the show_snapshots method in curator/cli_singletons/show.py, and then realized that I had left extra args in the method that were not being populated, so it was failing. All I had to do was remove those, and it ran.

I'm embarrassed, because I already fixed those once on a test platform, but didn't merge the code in properly, so it didn't make it into 5.5.3.

No worries @theuntergeek. It happens to the best of us.

Many thanks for the quick replies.

Curator 5.5.4 is released. The documentation site won't have the windows download links updated for at least 30 minutes (our doc builds are automated), but the RPM/DEB repos have 5.5.4, as does PyPi.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.