Logstash not giving output log events on console

Elastic Stack Logstash
1

Hi Everyone,
I am new to logstash. I am trying to run a logstash config file but facing a problem as discussed on the link :

Lateron in the post its mentioned that the issue got resolved as "It turns out my first logfile I am testing with is using old style line endings." Can anyone please explain what changes are needed to be made my logs file so that i can at least get an o/p on console.

Following are my files details :
1.) logstash-tutorial-dataset : contains logs
Sample logs in the file:

83.149.9.216 - - [04/Jan/2015:05:13:42 +0000] "GET /presentations/logstash-monitorama-2013/images/kibana-search.png HTTP/1.1" 200 203023 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"
83.149.9.216 - - [04/Jan/2015:05:13:42 +0000] "GET /presentations/logstash-monitorama-2013/images/kibana-dashboard3.png HTTP/1.1" 200 171717 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"
83.149.9.216 - - [04/Jan/2015:05:13:44 +0000] "GET /presentations/logstash-monitorama-2013/plugin/highlight/highlight.js HTTP/1.1" 200 26185 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"

2.) my filebeat.yml file configuration to read logs from above file and pass it to logstash :

filebeat.prospectors:

  • type: log
    paths:
    • /D:/shweta/ELK_poc/logstash-tutorial-dataset
      output.logstash:
      hosts: ["localhost:5044"]

3.) my logstash config file :

input {
beats {
port => "5044"
}
}
filter {
grok {
match => { "message" => "%{COMBINEDAPACHELOG}"}
}
}
output {
stdout { codec => rubydebug }
}

when i am trying to run command :
bin/logstash -f first-pipeline.conf --config.reload.automatic

its not giving any o/p. Just giving following :

Sending Logstash's logs to D:/shweta/ELK_poc/logstash-6.3.0/logs which is now configured via log4j2.properties
[2018-06-26T16:41:16,534][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2018-06-26T16:41:17,259][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"6.3.0"}
[2018-06-26T16:41:22,046][INFO ][logstash.pipeline ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>2, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[2018-06-26T16:41:23,157][INFO ][logstash.inputs.beats ] Beats inputs: Starting input listener {:address=>"0.0.0.0:5044"}
[2018-06-26T16:41:23,355][INFO ][logstash.pipeline ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x4df2d295 run>"}
[2018-06-26T16:41:23,386][INFO ][org.logstash.beats.Server] Starting server on port: 5044
[2018-06-26T16:41:23,519][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>}
[2018-06-26T16:41:23,941][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}

Here , there is no o/p. Can anyone help me figuring out the issue.

Thanks in advance
Shweta

2

Getting the exact same issue.

3

Issue got resolved when instead of using filebeat , i am directly giving log file path in the logstash conf file like :
input {
file {
path => "D:/shweta/ELK_poc/sample_input_log.txt"
start_position => "beginning"
sincedb_path => "NUL"
ignore_older => 0
}
}

filter {
grok {
match => { "message" => "%{COMBINEDAPACHELOG}"}
}
geoip {
source => "clientip"
}
}

output {
stdout { codec => rubydebug }
}

Though this is temporary fix without filebeat . If filebeat is to be used , need to figure out the issue, But for now its working fine for me removing filebeat.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.