Questions on ELK architecture

Elastic Stack Elasticsearch
1

Hello there

I have a cluster with two node, the second node added recently and only has Elasticsearch installed. Both are running on two different aws instances. Instance 1 has ELK installed and instance 2 has only Elasticsearch installed. My question here is -

  1. If the instance 1 goes down, is there a way (other than load balancing these two servers ) to view Kibana dashboard via instance 2? As of now there is only ES installed on ES 2.

  2. Apart from replicating the data from 1st ES node to 2nd ES node, what other function does adding the second node serve ? Is it only for replication? If yes how does that work incase there are issues with ES of instance 1

Thanks

2
  1. You can edit the config to point to the other one. But a load balancer is better.
  2. Replication = redundancy. It means you have more resources in the entire cluster, which improves performance.
3

Thank you for the response Mark!

  1. Could you please point me to the page, if any OR tell me the settings that need to be done on how to edit the config to point to other to access kibana from second node. I could not find the right doc for this

Regards

4

Look at elasticsearch.url - https://www.elastic.co/guide/en/kibana/current/settings.html

5

Thank you Aaron!

I referred the link and tried making changes to elasticsearch url (in kibana.yml of 1st instance) by adding host ip of second instance. So, the setting was :

elasticsearch.url: "http://:9200"

But I still cannot access kibana with this second ip and port 5601 ob browser. Telnet and ping both works fine from node 1 to node 2 and vice versa.

Could you please suggest what would have gone wrong? not sure if i'm missing something here. I 'm using 5.6.11 version

Thanks

6

Hi Aaron, Can i please get some help on this as it has become a road block to configure my prod cluster.

Thanks

7

That's not valid, it's missing the IP.

8

Hi Mark,

My apologies, that is something which I missed while replying to you. But my actual config does have the ip address and still doesn't seem to work.

9

So My first instance has ELK configured and the config is as below:

Kibana.yml:
server.port: 5601
server.host: "0.0.0.0"
elasticsearch.url: "http://<IP_ad_of_node2_running_on_different_awsinstance>:9200"
elasticsearch.preserveHost: true

elasticsearch.yml of node 1
cluster.name: NameEva
node.name: ThisIsNode1
network.host: 0.0.0.0
http.port: 9200

Node 2 (on separate instance) with only ES and here is the es config

cluster.name: NameEva
node.name: ThisIsNode2
network.host: 0.0.0.0
http.port: 9200
discovery.zen.ping.unicast.hosts: ["private ip of node1 instance"]
discovery.zen.minimum_master_nodes: 2

I'm able to access Kibana on port 5601 with the first node ip.

What am I missing here? Pls help

Thanks

10

What is the purpose of this configuration? If you are looking to achieve high availability you first and foremost will need at least 3 master-eligible Elasticsearch instances. You also most likely need two Kibana instances running on different hosts. Kibana can currently as far as I remember connect only to a single IP address, so have it connect to the local instance or use a load balancer.

11

Thank you Christian!

I'm aware of split brain scenario and the need for 3 nodes but for now i was trying to use Kibana through my second instance, just to check if I can access Kibana if node 1 is dead. Now I understand that the only option is to install Kibana on second node and use a load balance.

Regards

12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.