I have successfully set up an Elastic cluster with X-Pack and various Beats to simulate a production setting. However, I encountered some issues/questions while I was playing with the cluster.
My setup is like this:
- One master node running Ubuntu 16.04 (10.0.2.4) running Elasticsearch (to do query and stuff), Logstash, Kibana.
- Two nodes (10.0.2.5 - Ubuntu 16.04, 10.0.2.15 - CentOS 7) running Elasticsearch (to store replicas and shards) and all the Beats that deliver data to Logstash and Elasticsearch on 10.0.2.4.
My /etc/hosts file for all 3 machines contains
10.0.2.4 elastic-master.server
10.0.2.5 elastic-ubuntu.server
10.0.2.15 elastic-centos.server
On all 3 machines, elasticsearch.yml has this line
discovery.zen.ping.unicast.hosts: ["elastic-master.server", "elastic-ubuntu.server", "elastic-centos.server"]
-
Suppose I want to add a new node to the cluster. Do I edit all existing Elasticsearch nodes in the cluster to add the new node? What if the cluster already has a lot of nodes, say, 50 nodes. Do I have to edit all 50 nodes?
-
I tried to simulate a DDoS by sending 10000 logs/s from Filebeats on elastic-ubuntu and elastic-centos to Logstash on elastic-master. Some shards quickly died after that due to lack of free space. When I checked, there were about 3 million logs taking 6GB. What would be a good minimum amount of space for a production environment? Let's say I run a small website that receives several hundred to a thousand views/day and I want to monitor the traffic (This is just an example for me to get a feel of the resources needed).
-
The index rate was painfully low, maxing around 1k2/s. What should I do to increase it?