Hello, Forgive me if this is an issue that is repeated on here.
Originally, I had this working with what must be the exact same process. But, since I rebuilt the virtual machines something is no longer working and data is not making it from Filebeat to Logstash or Elasticsearch.
Also, this set up is to send snort logs.
Here are my versions:
- filebeat version 6.2.3 (amd64), libbeat 6.2.3
- kibana 6.2.2
- elasticsearch 6.2.2
- logstash 6.2.2
Everything starts up fine with no issues:
Logstash:
https://pastebin.com/raw/V7Jxv2VU
Elasticsearch:
https://pastebin.com/raw/mRS2iQDE
Kibana:
https://pastebin.com/raw/XwddpPsE
So, everything is up and in a single-node pointing to 192.168.0.53. Here is the logstash conf file:
input {
beats{
host => "192.168.0.53"
port => 5044
codec => plain {
charset => "US-ASCII"
}
}
}filter {
csv {
columns => ["datetime","sig_gen","sig_id","sig_rev","msg","proto","src_ip","src_port","dst_ip","dst_port","packet","category","priority","additional"]
}
mutate {
strip => ["datetime"]
}
date {
match => [ "datetime", "MM/dd/yy-HH:mm:ss.SSSSSS" ]
}if [priority] == "1" { mutate { add_field => { "severity" => "High" } } } if [priority] == "2" { mutate { add_field => { "severity" => "Medium" } } } if [priority] == "3" { mutate { add_field => { "severity" => "Low" } } } if [ids_proto] { if [ids_proto] =~ /^GPL/ { mutate { add_tag => [ "Snort-ET-sig" ] add_field => [ "ids_rule_type", "Emerging Threats" ] } } if [ids_proto] =~ /^ET/ { mutate { add_tag => [ "Snort-ET-sig" ] add_field => [ "ids_rule_type", "Emerging Threats" ] } } if "Snort-ET-sig" not in [tags] { mutate { add_tag => [ "Snort-sig" ] add_field => [ "ids_rule_type", "Snort" ] } } } if [src_ip] { geoip { source => "src_ip" target => "geoip" #database => "/etc/logstash/GeoLiteCity.dat" add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ] add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}" ] } mutate { convert => [ "[geoip][coordinates]", "float" ] } if ![geoip.ip] { if [dest_ip] { geoip { source => "dest_ip" target => "geoip" #database => "/etc/logstash/GeoLiteCity.dat" add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ] add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}" ] } mutate { convert => [ "[geoip][coordinates]", "float" ] } } } }
}
output {
elasticsearch {
hosts => [ "192.168.0.53:9200" ]
}
}
Finally, the entirety of the filebeat log:
2018-04-04T18:02:19.692+0100 INFO instance/beat.go:468 Home path: [/etc/filebeat] Config path: [/etc/filebeat] Data path: [/etc/filebeat/data] Logs path: [/etc/filebeat/logs] 2018-04-04T18:02:19.692+0100 INFO instance/beat.go:475 Beat UUID: 4f6379e1-5279-459c-805e-3115ef7234e6
I've tried to supply as much information as possible, but this is the exact same set up as I used before but something is going wrong.
Any ideas?
Thanks!
EDIT: FILEBEAT.YML
https://pastebin.com/raw/iYx0EidU