No output received in elasticsearch


(Lm Yj Q) #1

I've got the following debug info

18:40:43.230 [[main]>worker1] DEBUG logstash.pipeline - output received {"event"=>{"gender"=>"", "type"=>"report_video", "author_version"=>"None", "platform"=>"weixin", "_ext"=>{"content"=>"่ถ…่ตž๐Ÿ‘"}, "act"=>"video_comment", "register_at"=>"2017-02-11 19:25:22", "author_registered_at"=>nil, "@version"=>"1", "place"=>nil, "first_at"=>"2017-02-11 18:49:07", "udid"=>"871b0690b44eaf7c", "author_udid"=>"None", "doer_id"=>9472741, "author_zone"=>nil, "os"=>"and", "race"=>"user", "author_is_priest"=>nil, "timewindow"=>2490444, "author_os"=>"None", "version"=>"and-4.2.2", "age_zone"=>0, "@timestamp"=>2017-05-08T10:40:42.507Z, "logged_at"=>"2017-05-08 18:03:44", "post_id"=>nil, "user_id"=>9472741, "_task_id"=>482886, "_id"=>472745462, "author_id"=>7642061, "video_id"=>57752095}}

but kibana can't find any hits.

my config is as follow:

input {
redis {
host => "localhost"
data_type => "list"
key => "access"
type => "access"
}
redis {
host => "localhost"
data_type => "list"
key => "report_video"
type => "report_video"
}
}

filter {
json {
source => "message"
}

}

output {
elasticsearch {
hosts => "localhost:9200"
codec => "json"
index => "rawdata"
}
}

anyone have some idea about this? Thanks a lot!


(Magnus Bรคck) #2

Don't set the codec option for the elasticsearch output. Secondly, are you sure the time range you've chosen in Kibana covers the time of your events? Have you checked with ES's cat indices API that the index exists and that it contains documents? Have you looked in the Logstash log for clues?


(Lm Yj Q) #3

Thank you Magnus. I have fixed this issue.
The reason is that I have an "_id" field which is conflict with the metadata field has the same name.
The debug information appeared below. I found that after that.


(system) #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.