Logstash not able to read data from elasticsearch


(sushil) #1

Hello There,

I've a pre-populated elasticsearch and I am trying to use logstash to further analyse the data from elasticsearch. But whenever i run logstash, it never process any events, neither do i see anything in elasticsearch logs.

My conf file is:
input {

Read all documents from Elasticsearch matching the given query

elasticsearch {
hosts => ["localhost:9200"]
query => '{ "query": { "match": { "func_name": "MPMComponentAudioStopDecodingReal" } }, "sort": [ "_doc" ] }'
}
}

output {
stdout { codec => rubydebug }
}

when i run logstash in windows 10.
logstash.bat -f logstash-elastic.conf
Sending Logstash's logs to C:/Users/sushiku2/Downloads/ELK/logstash-5.4.0/logs which is now configured via log4j2.properties
[2017-07-22T08:22:21,972][INFO ][logstash.pipeline ] Starting pipeline {"id"=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>500}
[2017-07-22T08:22:22,629][INFO ][logstash.pipeline ] Pipeline main started
[2017-07-22T08:22:22,801][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}

I am facing same problem in Ubuntu system as well.
I have the default .yml file for both elasticsearch and logstash.

Any help in this is much appreciated.


(sushil) #2

I've found the solution for this problem when i ran logstash in debug mode and analysed its logs, looks like if you don't mention any index in input plugin for elastic search, it takes a default index of "logstash-*". However in my case the index was 77777. I modified my conf file and it solved the problem.

input {

Read all documents from Elasticsearch matching the given query

elasticsearch {
hosts => "localhost:9200"
index => "77777"
query => '{ "query": { "match": { "func_name": "MPMComponentAudioStopDecodingReal" } }, "sort": [ "_doc" ] }'
}
}

output {
stdout { codec => rubydebug }
}


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.