Elasticsearch One Server And Cluster Insert performance Almost the same! Why?

Recently tested single and cluster insert performance is almost the same, I don't really know why? Use java API TransportClient Bulk Insert.

version:
ElasticSearch 5.1.1

One Index:

  "settings" : {
    "index" : {
        "number_of_shards" : 5, 
        "number_of_replicas" : 0,
		"refresh_interval" : "30s"
    }
}

threads: 16 Bulk Insert:1000 docs(logs)
per log 44 field , per log for 1kb

single :
OS: Centos 7 8 cpus memery: 32GB
JVM Size Indexing Rate (/s)
4GB 6000~8000
8GB 6000~8000
16GB 6000~8000

cluster:
OS One: Centos 7 8 cpus memery: 32GB
master: JVM 4GB
date:JVM 16GB

OS Two: Centos 6.5 8 cpus memery: 32GB
date:JVM 16GB

Indexing Rate (/s) 6000~8000

I don't really know why?Bulk Insert docs performance is almost the same?

What does your client code look like?

Java API client code:

1: set clusterName、ipAndPorts
2:invoke initTransportClient method init connect
3:invoke bulkInsertDocument method,and set (index,type,json String list)

public class ElasticSearchBuilder {

private TransportClient client;
private String clusterName;
private String[] ipAndPorts;

public void setClusterName(String clusterName) {
    this.clusterName = clusterName;
}

public void setIpAndPorts(String[] ipAndPorts) {
    this.ipAndPorts = ipAndPorts;
}

public void initTransportClient() throws Exception {
    if (client != null) {
        client.close();
    }
    Settings settings = Settings.builder()
            .put("cluster.name", clusterName)
            .put("client.transport.sniff", true)
            .build();
    client = new PreBuiltTransportClient(settings);
    if (ipAndPorts == null || ipAndPorts.length < 1) {
        throw new NullPointerException("IP List is NULL");
    }
    for (String ipAndPort : ipAndPorts) {
        String[] ipAndPortString = ipAndPort.split(":");
        client.addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName(ipAndPortString[0]), Integer.parseInt(ipAndPortString[1])));
    }
}

public boolean bulkInsertDocument(String index, String type, List<String> jsonList) throws Exception {
    BulkRequestBuilder bulkRequest = client.prepareBulk();
    if (jsonList != null && jsonList.size() > 0) {
        for (String jsonData : jsonList) {
            bulkRequest.add(client.prepareIndex(index, type).setSource(jsonData));
        }
    }
    BulkResponse bulkResponse = bulkRequest.get();
    return !bulkResponse.hasFailures();
}

public void closeTransportClient() {
    client.close();
}

}

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.