Logstash crash / failed to connect to es cluster

Elastic Stack Logstash
1

I have an Issue that when i killed logstash and runned it again it it should pass 5k of messages and it does not. There are two errors there

One about no nodes avaible
Second :message=>"Got error to send bulk of actions: Failed to deserialize exception response from stream", :level=>:error}

Got logstash 1.5.4 with cluster of es on version 1.7.2
I use transport protocol for it.

Any help would be nice.

2

I'd recommend switching to the HTTP protocol. Doing so will almost certainly resolve this problem.

3

That is not an solution.
The problem exist somewhere and i need to fix it.
And if it is not my fault it needs to be fixed for future users

4

Of course it's a solution, just not the solution you like.

To increase your chances of getting a solution you do like, please quote complete error messages instead of saying e.g. "one about no nodes available".

5

One :

{:timestamp=>"2015-10-15T14:25:28.563000+0200", :message=>"Got error to send bulk of actions: None of the configured nodes are available: []", :level=>:error}
{:timestamp=>"2015-10-15T14:25:28.865000+0200", :message=>"Failed to flush outgoing items", :outgoing_count=>1, :exception=>"Java::OrgElasticsearchClientTransport::NoNodeAvailableException", :backtrace=>["org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(org/elasticsearch/client/transport/TransportClientNodesService.java:279)",

Second:

:timestamp=>"2015-10-15T14:25:18.658000+0200", :message=>"Got error to send bulk of actions: Failed to deserialize exception response from stream", :level=>:error}
6

And how's the health of the cluster? What does Logstash's elasticsearch output configuration look like?

7

As Magnus alluded, check your ES cluster. Chances are it's overloaded.

8

I checked the cluster and it is green. Its not about overload either since it is only input ( about 1k messages )

Config

output {
elasticsearch{
host => ["xxx.xxx.xxxx.xxx:9300","xxx.xxx.xxxx.xxx:9301","xxx.xxx.xxxx.xxx:9302"]
action => "index"
cluster => "elasticsearch"
index => "sergiusz"
protocol => "transport"
port => "9300-9305"
}

}

9

Is Logstash and Elasticsearch running the same Java version? If this is not the case it can cause problems, which is why HTTP will be the default protocol used by Logstash from version 2.0.

10

they run on same machine so Yes its same java i have readed about that already before i asked the question.

11

As i found out there was some error in different logstash . Did not saw that . All works now.

12

Is it possible to reveal the issue faced? I am facing similar issue and I am trying to eliminate the possibilities so as to identify the root cause.

regards