Hi Team,
I am having memory leak in elasticsearch java rest client.
since last 5-6 days i am analysing memory leak. i see there is leak in es rest client version 5.5
my client code is following:
**public void IndexComputeController (QueryIndexer computeIndex) {**
byte[] wikiArray;
String str = null;
IndexResponse response = null;
String filePath = computeIndex.getfilePath();
String index = computeIndex.getindex();
Long id = computeIndex.getId();
System.setProperty("io.netty.allocator.type", "unpooled");
// Creating pipeline for ingest attachment plugin
createPipeline ();
final CountDownLatch latch = new CountDownLatch(1);
try {
RestClient requester = RestClient.builder( new HttpHost ("192.168.*.*", 9300),
new HttpHost ("192.168.*.*", 9200)).build();
HttpEntity body = new NStringEntity("{" +
"\"data\":\"" + encoder(filePath) + "\"" + ",\n" +
"\"title\":\"" + filePath + "\"" +
"}", ContentType.APPLICATION_JSON);
HashMap<String,String> param = new HashMap<String,String>();
param.put("pipeline", "attachment");
Response httpresponse = requester.performRequest("PUT", "/"+index+"/"+"doc"+"/"+id, param, body);
requester.close();
} catch (Exception e) {
e.printStackTrace();
**}**
**public static boolean createPipeline(){**
RestClient requester = RestClient.builder( new HttpHost ("192.168.*.*", 9300),
new HttpHost ("192.168.*.*", 9200)).build();
HttpEntity body = new NStringEntity("{ \"description\" : \"Extract attachment information\", \n" +
" \"processors\": [\n" +
" {\n" +
" \"attachment\": {\n" +
" \"field\": \"data\",\n" +
" \"indexed_chars\": -1\n" +
" }\n" +
" },\n" +
" {\n" +
" \"set\": {\n" +
" \"field\": \"attachment.title\",\n" +
" \"value\": \"{{title}}\"\n" +
" }\n" +
" }\n" +
" ] \n" +
"}\n",ContentType.APPLICATION_JSON);
try {
requester.performRequest("PUT","/_ingest/pipeline/attachment", Collections.<String,String>emptyMap(), body);
} catch (IOException e) {
e.printStackTrace();
return false;
}
return true;
**}**
**public static String encoder(String path) throws Exception {**
String[] splitPath = path.split("\\.");
File file = new File(path);
if (!file.exists()) {
throw new Exception("File does not exist");
}
//Reading and encoding files
byte fileContent[] = Files.readAllBytes(file.toPath());
String result64 = javax.xml.bind.DatatypeConverter.printBase64Binary(fileContent);
return result64;
**}**
As soon i start uploading files (i am using ingest attachment plugin) when file reach to 117 its consume around 5GB memory and it say heap space is running out.
My ES server is running on different machine which is having ip address that i have mentioned in client side code.
Does anybody facing the same issue ?
How can i get rid of this memory leak.
Thank you very much in advance 