Hi all,
I want to upgrade my cluster from 6.2.4 to 6.5.4 (latest version)
Below are the components:
- 1 master node
 - 1 data node
 - 1 kibana node
 - 1 elastalert engine
 - 1 kafka node (Kafka version kafka_2.11)
 - 2 logstash nodes
 
Data flow:
- Logstash1 is collecting data in CEF format from endpoints. Producing the data to Kafka.
 - Logstash2 is consuming data from Kafka. Sending to Elasticsearch.
 - Kibana is having multiple dashboards for all log sources.
 - Elastalert is querying the data continuously for the configured rules.
 
My approach:
According to Elastic Support Matrix, 6.2.4 logstash supports the 6.5.4 version of Elasticsearch. So i am not gonna update logstash for now. Maybe, i'll update it in future, as using old versions reduces the functionality.
So, in this small architecture, I'll do below:
Stopping:
- Stop Logstash2 (Logs are being saved in topics till the time it updates) to prevent data loss when Elasticsearch would not be available to receive data.
 - Stop Kibana
 - Stop data node
 - Stop master node
 
Updating:
- Update Elasticsearch on master node
 - update Elasticsearch on data node
 - Update Kibana
 
Starting:
- Start master node
 - Start data node
 - Start Logstash2
 - Start Kibana node
 
Please help me with below questions:
- Is this approach good for this architecture?
 - Does this upgradation affects the data in the cluster (i.e.dashboards, elastalert rules, kibana objects etc.) ?
 - Do I need to upgrade Kafka too?
 - What are the recommended steps to upgrade such cluster?
 
Thanks in advance.