Unable to Scale Elastic Search Throughput in .Net

Elastic Stack Elasticsearch
1

I have created a .Net framework 4.5 application, I am trying to increase the overall throughput and response time of my application. Currently i am not able to scale further, despite the powerful hardware. Below are some details.

Currently my application through put is 720 Request/Sec, But same java application is giving me 1050 request/sec.

Elastic Cluster: Three Nodes
Windows Server: 8 cores 32 GB Ram.

What is weird is that my cpu consumption is under 30%, even under load and even after giving higher Tcp connection, i don't see that many outgoing connection. Below is my code, any suggestion are welcome.

  private static ElasticLowLevelClient _lowLevel;
var connectionPool = new SniffingConnectionPool(nodes);
            var config = new ConnectionConfiguration(connectionPool)
                .ConnectionLimit(10000)
                .EnableHttpCompression()
                .SniffOnConnectionFault(false)
                .SniffOnStartup(false)
                .EnableTcpKeepAlive(TimeSpan.FromMilliseconds(200), TimeSpan.FromMilliseconds(2000))
                .DisablePing();
            _lowLevel = new ElasticLowLevelClient(config); />```
2

Please format your code using </> in the editor toolbar, or enclose the code block in ```.

With such a high connection limit, I suspect the client is not efficiently able to reuse connections. The default connection limit on Windows in the client is set to 80; have you looked at benchmarking how changing the connection limit affects throughput?

Have you benchmarked how changing these values affects connection reuse and throughput? For

.EnableTcpKeepAlive(TimeSpan keepAliveTime, TimeSpan keepAliveInterval)

the Windows system defaults are

  • keepAliveTime is 2 hours
  • keepAliveInterval is 1 second

TCPView can be useful for monitoring TCP connections.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.