Hello,
I'm a relative novice who has managed to get the basic functions working between Elasticsearch and Kibana but am facing an issue with communication between Logstash and Elastic search. All data transfers are between instances on the local host.
I have tried to send the sample tutorial log file from Logstash into Elasticsearch via the command “bin/logstash –debug -f simple-pipeline.conf”.
It appears that logstash receives in the lines from the file but they are not received by elasticsearch. Does anyone have an idea why this is not happening as expected?
More details follow:
The simple-pipeline file contains the following settings
Input {
file {
path => "/var/log/InputLogs/logstash-tutorial.log"
start_position => beginning
}
}
Use standard Apache Log Grok filter
filter {
grok {
match => { "message" => "%{COMBINEDAPACHELOG}"}
}
}
output {
elasticsearch {
id => "tutor_log"
hosts => [ "localhost" ]
index => "tutor_index"
document_type => "tutor_type"
}
}
From the log messages, Logstash is receiving data from the input file “logstash-tutorial.log and passing it to Elasticsearch. However, Elasticsearch does not appear to be receiving this data. Elasticsearch is running fine and will accept files loaded directly to it.
The following lines are written in response to the logstash command. They continue in a loop until I close down the process.
13:11:15.586 [Api Webserver] INFO logstash.agent - Successfully started Logstash API endpoint {:port=>9600}
13:11:20.566 [Ruby-0-Thread-11: /usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:514] DEBUG logstash.pipeline - Pushing flush onto pipeline
13:11:25.565 [Ruby-0-Thread-11: /usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:514] DEBUG logstash.pipeline - Pushing flush onto pipeline
13:11:30.564 [Ruby-0-Thread-11: /usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:514] DEBUG logstash.pipeline - Pushing flush onto pipeline
13:11:35.565 [Ruby-0-Thread-11: /usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:514] DEBUG logstash.pipeline - Pushing flush onto pipeline