ES 1.6 can't query for existing snapshots

Elastic Stack Elasticsearch
1

Hi All !

I have been playing today with ES snapshots feature executing specific indices snapshots directly using REST API with curl and with curator.

After several tests like create/delete snapshot for index, and the same for a larger indices group I run into strange issue when I am unable to query for snapshot list within my repository ( 'my_backup' ).

Tried restarting all the nodes but issue seems to be persistent.
The same happens regardless currenty picked master node.

Here is my outputs :

My current repo :

    root@kibana:~# curl -XGET 'http://localhost:9700/_snapshot/my_backup' | python -m json.tool
 {
        "my_backup": {
            "settings": {
                "location": "/mnt/es_backup"
            },
            "type": "fs"
        }
    }

My current test snapshot _1 :

root@kibana:~# curl -XGET 'http://localhost:9700/_snapshot/my_backup/snapshot_1' | python -m json.tool
{
    "snapshots": [
        {
            "duration_in_millis": 91,
            "end_time": "2015-07-15T12:21:01.580Z",
            "end_time_in_millis": 1436962861580,
            "failures": [],
            "indices": [
                "logstash-2015.06.18"
            ],
            "shards": {
                "failed": 0,
                "successful": 5,
                "total": 5
            },
            "snapshot": "snapshot_1",
            "start_time": "2015-07-15T12:21:01.489Z",
            "start_time_in_millis": 1436962861489,
            "state": "SUCCESS"
        }
    ]
}

Here's the problem - I am unable to query for snapshots stored under my_backup :

root@kibana:~# curl -XGET 'http://localhost:9700/_snapshot/my_backup/_all' | python -m json.tool
{
    "error": "RemoteTransportException[[vmtest3_node][inet[/192.168.106.29:9300]][cluster:admin/snapshot/get]]; nested: ElasticsearchParseException[Failed to derive xcontent from (offset=0, length=0): []]; ",
    "status": 400
}

Same with curator :

root@kibana:~# curator --debug show snapshots --all-snapshots --repository my_backup
2015-07-15 14:33:57,895 INFO      curator.cli.snapshot_selection              snapshots:50   Job starting: show snapshots
2015-07-15 14:33:57,895 DEBUG          curator.cli.utils             get_client:110  kwargs = {'url_prefix': u'', 'http_auth': None, 'dry_run': False, 'loglevel': u'INFO', 'host': u'localhost', 'logformat': u'default', 'timeout': 30, 'debug': True, 'use_ssl': False, 'logfile': None, 'master_only': False, 'port': 9200}
2015-07-15 14:33:57,896 DEBUG         urllib3.util.retry               from_int:155  Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0)
2015-07-15 14:33:57,896 INFO      urllib3.connectionpool              _new_conn:188  Starting new HTTP connection (1): localhost
2015-07-15 14:33:57,899 DEBUG     urllib3.connectionpool          _make_request:362  "GET / HTTP/1.1" 200 340
2015-07-15 14:33:57,899 INFO               elasticsearch    log_request_success:57   GET http://localhost:9200/ [status:200 request:0.003s]
2015-07-15 14:33:57,899 DEBUG              elasticsearch    log_request_success:59   > None
2015-07-15 14:33:57,899 DEBUG              elasticsearch    log_request_success:60   < {
  "status" : 200,
  "name" : "kibana_node",
  "cluster_name" : "ES_CLUSTER",
  "version" : {
    "number" : "1.6.0",
    "build_hash" : "cdd3ac4dde4f69524ec0a14de3828cb95bbb86d0",
    "build_timestamp" : "2015-06-09T13:36:34Z",
    "build_snapshot" : false,
    "lucene_version" : "4.10.4"
  },
  "tagline" : "You Know, for Search"
}

2015-07-15 14:33:57,900 DEBUG          curator.cli.utils          check_version:90   Detected Elasticsearch version 1.6.0
2015-07-15 14:33:57,900 DEBUG         urllib3.util.retry               from_int:155  Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0)
2015-07-15 14:33:57,900 INFO      urllib3.connectionpool              _get_conn:223  Resetting dropped connection: localhost
2015-07-15 14:33:58,056 DEBUG     urllib3.connectionpool          _make_request:362  "GET /_snapshot/my_backup/_all HTTP/1.1" 400 217
2015-07-15 14:33:58,056 WARNING            elasticsearch       log_request_fail:76   GET /_snapshot/my_backup/_all [status:400 request:0.156s]
2015-07-15 14:33:58,056 INFO               elasticsearch       log_request_fail:84   > None
2015-07-15 14:33:58,056 ERROR          curator.api.utils          get_snapshots:240  Unable to find all snapshots in repository: my_backup
ERROR. No snapshots found in Elasticsearch.

Would appreciate all the help.

Best Regards,
maq