I am currently deploying elasticsearch(logsearch) using bosh-releases
I am using logsearch boshrelease v 208.0.0.
This deployment creates folowing instances in AWS
- elasticsearch_data/1
- kibana/0
- ingestor/0
- elasticsearch_data/2
- ls-router/0
- elasticsearch_master/0
- elasticsearch_data/0
- maintenance/0
- cluster_monitor/0 .
Instances successfully created were: elasticsearch_master/0, maintenance/0, cluster_monitor.
This current deployment is failing while creating elasticsearch_data/0 instance.
Error message:
Updating instance elasticsearch_data: elasticsearch_data/<instance-id>(0) (canary) (00:05:34)
L Error: Action Failed get_task: Task <task-id>result: 1 of 1 post-start scripts failed. Failed Jobs: elasticsearch.
Task 1111 | 15:15:38 | Error: Action Failed get_task: Task <task-id> result: 1 of 1 post-start scripts failed. Failed Jobs: elasticsearch
I have checked both elasticsearch_master logs(/var/vcap/sys/log/elasticsearch/elasticsearch.stdout.log) and this was only WARN message:
/[WARN ][r.suppressed ] path: /_bulk, params: {}
org.elasticsearch.transport.NodeNotConnectedException: [elasticsearch_data/0][0.0.0.0:9300] Node not connected.
I have checked elasticsearch_data/0 log (/var/vcap/sys/log/elasticsearch/elasticsearch.stdout.log)
data node was able to add elasticsearch_master
[ elasticsearch_data/0] detected_master {elasticsearch_master/0}{4IQmYSorQqC3QxjCZ5TsD
g}{NERLgl7AQgKEAWs6HNnhAw}{10.249.1.121}{10.249.1.121:9300}, added {{elasticsearch_master/0}{4IQmYSorQqC3QxjCZ5TsDg}{NERLgl7AQgKEAWs6HNnhAw}{10.2
49.1.121}{10.249.1.121:9300},}, reason: zen-disco-receive(from master [master {elasticsearch_master/0}{4IQmYSorQqC3QxjCZ5TsDg}{NERLgl7AQgKEAWs6HN
nhAw}{10.249.1.121}{10.249.1.121:9300} committed version [1
I have checked elasticsearch_data/0 (/var/vcap/var/vcap/sys/log/elasticsearch/post-start.stdout.log). which gives health and status of the cluster
body: {"cluster_name":"logsearch","status":"yellow","timed_out":false,"number_of_nodes":2,"number_of_data_nodes":1,"active_primary_shards":18,"ac
tive_shards":18,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":18,"delayed_unassigned_shards":0,"number_of_pending_tasks":0,"n
umber_of_in_flight_fetch":0,"task_max_waiting_in_queue_millis":0,"active_shards_percent_as_number":50.0}
status:
In the deployment I have actually specified to disable_post_start: false
jobs:
- consumes:
elasticsearch:
from: elasticsearch_master
health:
disable_post_start: true
timeout: 600
log_level: DEBUG
Can you give me some information or direction on why it is failing?.
let me know if you any additional information you need.
Thanks,
Nirish M