Create Cluster with Nodes in Different Server

Is there any documentation that mention how this is done. All the webpages that I can find out there cannot work.

Is this feature still available on elasticsearch? If yes. Can someone point me to the right place?

https://www.elastic.co/guide/en/elasticsearch/reference/2.2/modules-discovery-zen.html and https://www.elastic.co/guide/en/elasticsearch/guide/master/_add_failover.html should be all you need.

Also https://www.elastic.co/guide/en/elasticsearch/guide/master/important-configuration-changes.html

These are my settings on the two different server

Server 1
cluster.name: ClusterTest
node.name: node-1
node.master: true
network.host: 192.168.1.109

transport.tcp.port: 9300
discovery.zen.ping.unicast.hosts: ["192.168.1.109", "192.168.1.94:9200"]

Server 2
cluster.name: ClusterTest
node.name: node-2
node.master: true
network.host: 192.168.1.109

transport.tcp.port: 9300
discovery.zen.ping.unicast.hosts: ["192.168.1.109", "192.168.1.94:9200"]

As you have 2 master eligible nodes, have you set minimum_master_nodes to 2 in the cluster?

This should connect on port 9300, not 9200 (which is for HTTP).

Not yet. Which server should I set the minimum_master_nodes to 2?

On both servers. It is a cluster wide setting.

My latest configuration. Still cannot work

cluster.name: ClusterTest
node.name: node-1
node.master: true
network.host: 192.168.1.109
transport.tcp.port: 9300
discovery.zen.ping.unicast.hosts: ["192.168.1.94", "192.168.1.109:9200"]
discovery.zen.minimum_master_nodes: 2

cluster.name: ClusterTest
node.name: node-5
network.host: 192.168.1.94
node.master: true
transport.tcp.port: 9300
discovery.zen.ping.unicast.hosts: ["192.168.1.94", "192.168.1.109:9200"]
discovery.zen.minimum_master_nodes: 2

You should re-read Christian's previous comments.

This should specify port 9300.

What should I see if it is working.

Below is my latest configuration. I am hoping to see data being accessible from both servers

cluster.name: ClusterTest
node.name: node-1
node.master: true
network.host: 10.0.0.52
transport.tcp.port: 9200
discovery.zen.ping.unicast.hosts: ["10.0.0.52", "10.0.0.53:9300"]
discovery.zen.minimum_master_nodes: 2

ClusterTest:
node.name: node-5
network.host: 10.0.0.53
#node.master: false
transport.tcp.port: 9200
discovery.zen.ping.unicast.hosts: ["10.0.0.53", "10.0.0.52:9300"]
discovery.zen.minimum_master_nodes: 2

If it works then _cat/nodes will list more than one.

Why did you change this to 9200? This is supposed to be 9300.

Is this correct now?

cluster.name: ClusterTest
node.name: node-1
node.master: true
network.host: 10.0.0.52
transport.tcp.port: 9300
discovery.zen.ping.unicast.hosts: ["10.0.0.52", "10.0.0.53:9300"]
discovery.zen.minimum_master_nodes: 2

cluster.name: ClusterTest
node.name: node-5
network.host: 10.0.0.53
#node.master: false
transport.tcp.port: 9300
discovery.zen.ping.unicast.hosts: ["10.0.0.52", "10.0.0.52:9300"]
discovery.zen.minimum_master_nodes: 2

I wouldn't bother even declaring transport.tcp.port, just leave it out.

Do you have some working samples? Nothing online seems to work on my systems

With 2.X all you need is to set these;

cluster.name: 
node.name: 
network.host: 
discovery.zen.ping.unicast.hosts: [""]
discovery.zen.minimum_master_nodes: 

It looks like it's a 2-node cluster and you have discovery.zen.minimum_master_nodes set to 2, this means the second node, node-5, must have node.master set to true so you'll have two master+data node in the cluster. Without the second master, it won't work.

Alternatively, for testing, you can set discovery.zen.minimum_master_nodes to 1 for both node-1 and node-5 and as long as node-1 has node.master set to true, the cluster should go up with no problem.

Still does not work.

Giving up. Concluding that multiple node across different server is supported

You're right, it is supported.
Maybe you have a firewall or something in place?