Logstash unable to send events to Elasticsearch


(Kishore) #1

I have upgraded the logstash version from 1.4.2 and elasticsearch version from 1.5.0. to 2.1.1 Up gradation was successful, but facing issue while sending events from logstash to Elasticsearch.

Logstash is unable to send the events to elasticsearch?

Kindly help in solving this issue.


(Christian Dahlqvist) #2

What does your configuration look like?


(Kishore) #3

My output configuration
output {
elasticsearch {
hosts => [ "22.22.22.222:9200" ]
#protocol => "http"
}


(Christian Dahlqvist) #4

Do you see anything in the logs/output? Have you tried starting it in verbose or debug mode? Have you verified that Logstash is actually receiving events to be sent to Elasticsearch, e.g. by using the stdout plugin, so that it is not a problem with one of the inputs or filters?


(Kishore) #5

I have started in verbose mode, yes i have verified that logstash is receiving events.

Do we require to change any configuration changes at Elasticsearch node?


(Kishore) #6

I am using redis as input source, below i am sharing input configuration

input {
redis {
host => "ip-11-111-111-111.ec2.internal"
port => "7000"
data_type => "list"
key => "syslog"
type => "redis-input"
}
}

Do i have made any configuration changes here?


(Christian Dahlqvist) #7

Which version of Logstash are you using? Was there anything in the verbose output that indicate a problem? Can you show us a sample event output through the stdout plugin with rubydebug codec?


(Kishore) #8

logstash version is 2.1.1 and getiing below warning

"Redis connection problem {:exception=>#Redis::CommandError: CROSSSLOT Keys in request don't hash to the same slot>, :level=>:warn}"


(Kishore) #9

I got below error when i tried sample event output through the stdout plugin with rubydebug codec.

"Failed action. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"logstash-2016.01.29", :_type=>"logs", :_routing=>nil}, #<LogStash::Event:0x2efa5a8b @metadata_accessors=#<LogStash::Util::Accessors:0x38ad4c74 @store={}, @lut={}>, @cancelled=false, @data={"message"=>"This is a sample", "@version"=>"1", "@timestamp"=>"2016-01-29T13:25:48.826Z", "host"=>"ip-20-334-233-111.ec2.internal"}, @metadata={}, @accessors=#<LogStash::Util::Accessors:0x7021aced @store={"message"=>"This is a sample", "@version"=>"1", "@timestamp"=>"2016-01-29T13:25:48.826Z", "host"=>"ip-20-334-233-111.ec2.internal"}, @lut={"host"=>[{"message"=>"This is a sample", "@version"=>"1", "@timestamp"=>"2016-01-29T13:25:48.826Z", "host"=>"ip-20-334-233-111.ec2.internal"}, "host"], "type"=>[{"message"=>"This is a sample", "@version"=>"1", "@timestamp"=>"2016-01-29T13:25:48.826Z", "host"=>"ip-20-334-233-111.ec2.internal"}, "type"]}>>], :response=>{"index"=>{"_index"=>"logstash-2016.01.29", "_type"=>"logs", "_id"=>nil, "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"Failed to parse mapping [default]: Mapping definition for [geoip] has unsupported parameters: [path : full]", "caused_by"=>{"type"=>"mapper_parsing_exception", "reason"=>"Mapping definition for [geoip] has unsupported parameters: [path : full]"}}}}, :level=>:warn}"


(system) #10