We're currently testing our new ES solution and have found that it isn't reusing existing connections. The HTTP total_opened value (GET _nodes/stats/http) is consistently increasing as we do our indexing/run queries. Current values for our 3 nodes are below:
The current_open values move between 0 and 2 while total_opened increases. Also, node-1 looks to be receiving a lot higher percentage of traffic.
If you see a very large total_opened number that is constantly increasing, that is a sure sign that one of your HTTP clients is not using keep-alive connections. Persistent, keep-alive connections are important for performance, since building up and tearing down sockets is expensive (and wastes file descriptors). Make sure your clients are configured appropriately.
I've updated our index settings to include KeepAlive ("keepAlive": "true") but it's made no difference.
We're using the .Net NEST library, a StaticConnectionPool, ES version 2.3 on Windows Server 2012 machines. The client is running as a singleton instance.
Does anyone know any ideas on how we can force our solution to use existing connections? Or what the repercussions of this might be?
Thanks in advance