filebeat.yml file
filebeat.prospectors:
type: log
paths:
"/home/ec2-user/logs/sas_logs/*.log"
output.logstash:
hosts: ["localhost:5044"]
logstash.conf file
input {
beats {
port => "5044"
}
}
filter {
grok {
match => { "message" => "%{TIMESTAMP_ISO8601:time_stamp}\s+%{LOGLEVEL:loglevel}\s+[%{NUMBER:num}]\s+:%{USERNAME:user}\s+-\s+%{GREEDYDATA:msg}" }
}
}
output {
elasticsearch {
hosts => "http://localhost:9200 "
index => "a3_logs"
}
stdout {
codec => rubydebug
}
}
log format: 2018-07-09T16:20:30,799 INFO [00000006] :sasinst - NOTE: Copyright (c) 2016 by SAS Institute Inc., Cary, NC, USA.
When I run logstash, it is stuck at Pipeline Started. I'm not able to understand the issue.
Can anyone please help me ??
pierhugues
(Pier-Hugues Pellerin)
August 8, 2018, 6:02pm
2
Hello @Vijay_Chander , by looking at your filebeat configuration, I believe you have an indentation issue in your configuration and you also need to enable the logstash output.
filebeat.prospectors:
- type: log
paths:
- "/home/ec2-user/logs/sas_logs/*.log"
output.logstash:
enabled: true
hosts: ["localhost:5044"]
Also starting filebeat in debug mode should give us a a better idea is there is others issues, you can do that by running the following command:
./filebeat -v -e -d "*" -c ./myconfig.yml
1 Like
Hi @pierhugues ,
I've tried as you suggested but still no luck. Below is my filebeat log
2018-08-09T05:32:11.870Z DEBUG [input] log/input.go:175 input states cleaned up. Before: 5, After: 5, Pending: 0
logstash log
[2018-08-09T05:27:28,029][INFO ][logstash.inputs.beats ] Beats inputs: Starting input listener {:address=>"0.0.0.0:5044"}
[2018-08-09T05:27:28,141][INFO ][logstash.pipeline ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x1e67cb22 run>"}
[2018-08-09T05:27:28,187][INFO ][org.logstash.beats.Server] Starting server on port: 5044
[2018-08-09T05:27:28,322][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2018-08-09T05:27:28,614][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
Can you please suggest any other way??
pierhugues
(Pier-Hugues Pellerin)
August 9, 2018, 12:24pm
4
@Vijay_Chander lets concentrate on the Filebeat side, If you want filebeat with the options I've said in my previous comment the log should be a lot bigger. Can you include a more complete log?
system
(system)
Closed
September 6, 2018, 12:33pm
5
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.