I have a multi node cluster with 3 masters, 4 data nodes, 2 ingest nodes.
My cluster is working and I am blocking port 9300 on one of the data or master nodes. To block port 9300 which is used communicating between the nodes I am using below command :
sudo ufw deny 9300.
I can see in firewall status that 9300 port has been denied , but still I can see that search, post requests are being served by that node. I have enabled trace logging and can see in logs that every 30 seconds master is pinging data nodes and getting response.
Is this expected behaviour ? How Elastic is serving request even if port has been blocked.