F5 load balancer SSL_ERROR_SYSCALL, errno 104 with Elasticsearch cluster

Elastic Stack Elasticsearch
1

We have configured F5 LB in front of Elasticsearch nodes cluster with re-encrypt of SSL traffic to the nodes. Nodes have SSL enabled on http. Direct communication to nodes i.e. API (curl) or sending data over https on port 9200 works, however communication from F5 LB gives SSL_ERROR_SYSCALL, errno 104.
The reason of implementing F5 LB in front of the elasticsearch nodes is for managing configuration over HA single endpoint and external monitoring of the elastic cluster over HA single endpoint. (HA Elastic Cluster with more then 6 nodes spread over two availability zones in two data centers)
We can't find any documentation if this configuration is supported by Elastic since all related questions have "it depends" as answer. i.e. F5 load balancer attributes/configuration with Elasticsearch cluster
From only place where Load Balancers are mentioned is the Elastic Cloud which lists that HTTP mode should be used as "unencrypted" communication between the LB and the nodes.
Load balancers | Elastic Cloud Enterprise Reference [3.6] | Elastic

  • HTTP: Use HTTP mode for ports 9200/9243 (HTTP traffic to clusters) and also for ports 12400/12443 (adminconsole traffic). Make sure that all load balancers or proxies sending HTTP traffic to deployments hosted on Elastic Cloud Enterprise are sending HTTP/1.1 traffic.

I would really appreciate is someone can share functional solution if re-encrypt of traffic between LB backend and nodes can work as Elastic is encouraging to use the nodes as their ingest data endpoints for better load balancing designed by Elastic but when it comes to configuration management and monitoring from external we can't just loop over list of hosts.

2

A properly configured HTTP(S) load balancer between clients and Elasticsearch should work just fine. The traffic between clients and ES is just regular HTTP(S), there's no magic here.

Therefore, if your load balancer is not working then it's almost certainly just not configured correctly. Unfortunately you're unlikely to get much help with configuring F5 here, this forum's focus is Elasticsearch but there's not really an ES question to answer here. You likely need to open a support case with F5.

3

Thank you for short clarification that properly configured LB should work so we will continue to investigate if we can find any error within the configuration.
My main remark here is why HTTP is recommended as to be used when using Elastic Cloud behind Load Balancer in official Elastic documentation?

4

I'm not sure I understand your question. The docs say to use HTTP because that's the protocol Elasticsearch uses.

5

The docs you linked are specifically about Elastic Cloud Enterprise (ECE), not Elasticsearch in general. ECE comes with a support contract, so if you have a question about ECE it would be best to open a support case.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.