Hi All !
I have been playing today with ES snapshots feature executing specific indices snapshots directly using REST API with curl and with curator.
After several tests like create/delete snapshot for index, and the same for a larger indices group I run into strange issue when I am unable to query for snapshot list within my repository ( 'my_backup' ).
Tried restarting all the nodes but issue seems to be persistent.
The same happens regardless currenty picked master node.
Here is my outputs :
My current repo :
root@kibana:~# curl -XGET 'http://localhost:9700/_snapshot/my_backup' | python -m json.tool
{
"my_backup": {
"settings": {
"location": "/mnt/es_backup"
},
"type": "fs"
}
}
My current test snapshot _1 :
root@kibana:~# curl -XGET 'http://localhost:9700/_snapshot/my_backup/snapshot_1' | python -m json.tool
{
"snapshots": [
{
"duration_in_millis": 91,
"end_time": "2015-07-15T12:21:01.580Z",
"end_time_in_millis": 1436962861580,
"failures": [],
"indices": [
"logstash-2015.06.18"
],
"shards": {
"failed": 0,
"successful": 5,
"total": 5
},
"snapshot": "snapshot_1",
"start_time": "2015-07-15T12:21:01.489Z",
"start_time_in_millis": 1436962861489,
"state": "SUCCESS"
}
]
}
Here's the problem - I am unable to query for snapshots stored under my_backup :
root@kibana:~# curl -XGET 'http://localhost:9700/_snapshot/my_backup/_all' | python -m json.tool
{
"error": "RemoteTransportException[[vmtest3_node][inet[/192.168.106.29:9300]][cluster:admin/snapshot/get]]; nested: ElasticsearchParseException[Failed to derive xcontent from (offset=0, length=0): []]; ",
"status": 400
}
Same with curator :
root@kibana:~# curator --debug show snapshots --all-snapshots --repository my_backup
2015-07-15 14:33:57,895 INFO curator.cli.snapshot_selection snapshots:50 Job starting: show snapshots
2015-07-15 14:33:57,895 DEBUG curator.cli.utils get_client:110 kwargs = {'url_prefix': u'', 'http_auth': None, 'dry_run': False, 'loglevel': u'INFO', 'host': u'localhost', 'logformat': u'default', 'timeout': 30, 'debug': True, 'use_ssl': False, 'logfile': None, 'master_only': False, 'port': 9200}
2015-07-15 14:33:57,896 DEBUG urllib3.util.retry from_int:155 Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0)
2015-07-15 14:33:57,896 INFO urllib3.connectionpool _new_conn:188 Starting new HTTP connection (1): localhost
2015-07-15 14:33:57,899 DEBUG urllib3.connectionpool _make_request:362 "GET / HTTP/1.1" 200 340
2015-07-15 14:33:57,899 INFO elasticsearch log_request_success:57 GET http://localhost:9200/ [status:200 request:0.003s]
2015-07-15 14:33:57,899 DEBUG elasticsearch log_request_success:59 > None
2015-07-15 14:33:57,899 DEBUG elasticsearch log_request_success:60 < {
"status" : 200,
"name" : "kibana_node",
"cluster_name" : "ES_CLUSTER",
"version" : {
"number" : "1.6.0",
"build_hash" : "cdd3ac4dde4f69524ec0a14de3828cb95bbb86d0",
"build_timestamp" : "2015-06-09T13:36:34Z",
"build_snapshot" : false,
"lucene_version" : "4.10.4"
},
"tagline" : "You Know, for Search"
}
2015-07-15 14:33:57,900 DEBUG curator.cli.utils check_version:90 Detected Elasticsearch version 1.6.0
2015-07-15 14:33:57,900 DEBUG urllib3.util.retry from_int:155 Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0)
2015-07-15 14:33:57,900 INFO urllib3.connectionpool _get_conn:223 Resetting dropped connection: localhost
2015-07-15 14:33:58,056 DEBUG urllib3.connectionpool _make_request:362 "GET /_snapshot/my_backup/_all HTTP/1.1" 400 217
2015-07-15 14:33:58,056 WARNING elasticsearch log_request_fail:76 GET /_snapshot/my_backup/_all [status:400 request:0.156s]
2015-07-15 14:33:58,056 INFO elasticsearch log_request_fail:84 > None
2015-07-15 14:33:58,056 ERROR curator.api.utils get_snapshots:240 Unable to find all snapshots in repository: my_backup
ERROR. No snapshots found in Elasticsearch.
Would appreciate all the help.
Best Regards,
maq