How to take elasticsearch index backup to HDFS

[sundarcse1216 (Sundar)]

Hi All,

Good day all..

I want to take elasticsearch index backup and store into HDFS. Can anyone please help me?

Thanks,
Sundar.

[Christian_Dahlqvist]

Have you looked at the snapshot and restore functionality and the HDFS snapshot plugin?

[sundarcse1216 (Sundar)]

Thanks for your response.

Yes we installed repository-hdfs plugin in all node in the cluster and restarted.

When we send snapshot request it throwing error response.

Request:

     PUT _snapshot/my_hdfs_repository
         {
         "type": "hdfs",
         "settings": {
         "uri": "hdfs://xxxxxxxx:12345/",
         "path": "/dev/test/snapshot",
         "security.principal": "abc@TEST-xyv.COM"
     }
  }

Response:

{ "error" : { "root_cause" : [ { "type" : "repository_exception", "reason" : "[my_hdfs_repository] failed to create repository" } ], "type" : "repository_exception", "reason" : "[my_hdfs_repository] failed to create repository", "caused_by" : { "type" : "unchecked_i_o_exception", "reason" : "Cannot create HDFS repository for uri [hdfs://yyyyyyy:12345/ "caused_by" : { "type" : "i_o_exception", "reason" : "Failed on local exception: java.io.IOException: org.apache.h s : local host is: \"xxxxxxxx.server.net/xx.yyyy.zz.qq\"; destination host is "caused_by" : { "type" : "i_o_exception", "reason" : "**org.apache.hadoop.ipc.RpcException**: RPC response exceeds m "caused_by" : { "type" : "rpc_exception", "reason" : "RPC response exceeds maximum data length" } } } } }, "status" : 500 }

Please help me to resolve this issue. Thanks in advance.