Hi,
A couple of days now i have problem of providing logs on kibana. (filebeat - winlogbeat)
All these logs are parsing through logstash.
Logstash error logs:
[2019-07-11T12:34:12,246][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 503 ({"type"=>"unavailable_shards_exception", "reason"=>"[filebeat-7.0.1-2019.07.08][0] primary shard is not active Timeout: [1m], request: [BulkShardRequest [[filebeat-7.0.1-2019.07.08][0]] containing [6] requests]"})
[2019-07-11T12:34:12,246][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 503 ({"type"=>"unavailable_shards_exception", "reason"=>"[filebeat-7.0.1-2019.07.08][0] primary shard is not active Timeout: [1m], request: [BulkShardRequest [[filebeat-7.0.1-2019.07.08][0]] containing [6] requests]"})
[2019-07-11T12:34:12,246][INFO ][logstash.outputs.elasticsearch] Retrying individual bulk actions that failed or were rejected by the previous bulk request. {:count=>6}
[2019-07-11T12:34:13,470][WARN ][logstash.outputs.elasticsearch] Marking url as dead. Last error: [LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError] Elasticsearch Unreachable: [http://atdevxhv03.emea.nsn-net.net:9200/][Manticore::SocketTimeout] Read timed out {:url=>http://atdevxhv03.emea.nsn-net.net:9200/, :error_message=>"Elasticsearch Unreachable: [http://atdevxhv03.emea.nsn-net.net:9200/][Manticore::SocketTimeout] Read timed out", :error_class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError"}
[2019-07-11T12:34:13,471][ERROR][logstash.outputs.elasticsearch] Attempted to send a bulk request to elasticsearch' but Elasticsearch appears to be unreachable or down! {:error_message=>"Elasticsearch Unreachable: [http://atdevxhv03.emea.nsn-net.net:9200/][Manticore::SocketTimeout] Read timed out", :class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError", :will_retry_in_seconds=>2}
[2019-07-11T12:34:14,821][ERROR][logstash.outputs.elasticsearch] Attempted to send a bulk request to elasticsearch, but no there are no living connections in the connection pool. Perhaps Elasticsearch is unreachable or down? {:error_message=>"No Available connections", :class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError", :will_retry_in_seconds=>2}
[2019-07-11T12:34:15,472][ERROR][logstash.outputs.elasticsearch] Attempted to send a bulk request to elasticsearch, but no there are no living connections in the connection pool. Perhaps Elasticsearch is unreachable or down? {:error_message=>"No Available connections", :class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError", :will_retry_in_seconds=>4}
[2019-07-11T12:34:15,735][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://atdevxhv03.emea.nsn-net.net:9200/"}
[root@atdevxhv03 logstash]#
And this is the test of the filebeat:
[root@atdevxhv03 logstash]# filebeat test output
logstash: atdevxhv03.emea.nsn-net.net:5044...
connection...
parse host... OK
dns lookup... OK
addresses: 10.158.67.175
dial up... OK
TLS... WARN secure connection disabled
talk to server... OK
For one reason no filebeat and winlogbeat indices are creating.
Do you have any idea?
Best Regards,
Thanos