Error on logstash, can't connect to elasticsearch

Elastic Stack Logstash
1

Hi

Currently I am using logstash forwarder , logstash and elasticsearch to monitor my syslogs on various machines. It was working fine but it stopped working all of sudden. I noticed my logstash had stopped working.

When i tried restarting my logstash, it displayed following error in logstash.logs. It is currently in start condition but not being able to connect with elasticsearch.

message=>"Got error to send bulk of actions to elasticsearch server at localhost : blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];

Does anybody know how to debug the issue.

My logstash config file is following:

output {
  elasticsearch { host => localhost }
  stdout { codec => rubydebug }
}

Elasticsearch cluster is pretty basic , one thing i have done is that initially i had started out with one machine and recently i converted it into two machines but my hostname has remained "localhost" for elasticsearch.

Thanks!

2

This means your ES cluster is not in a healthy state.

What does curl ES_IP:9200/_cat/health show?

3

Hi Mark,

Thanks for the reply, i checked the elasticsearch cluster is working correctly. If i try to insert something directly it gets inserted and i can view it in kibana as well. Following is the output of the command:

curl localhost:9200/_cluster/health?pretty
{
  "cluster_name" : "elasticsearch",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 2,
  "number_of_data_nodes" : 2,
  "active_primary_shards" : 466,
  "active_shards" : 932,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0
}
4

Try adding this to the ES output config;

protocol => http
5

Hi

I added it but it didn't work.
In the end , i had to clear my logstash.log history and followed the following link which made it work:

After this it started working.