I am knew to ELK and I can't see the logs coming from port 5002 on kibana. Can anyone please help me configure logstash to send the logs to kibana?
I know the logs are coming:
nettec@Ubuntu-ELK:/etc/logstash/conf.d$ nc -l -u 10.0.1.4 5002
<134>Sep 28 10:57:43 filterlog: 5,,,1000000103,igb3,match,block,in,4,0x0,,64,0,0,none,17,udp,149,177.85.233.177,255.255.255.255,5678,5678,129<134>Sep 28 10:57:43 filterlog: 5,,,1000000103,igb3,match,block,in,4,0x0,,64,0,0,none,17,udp,160,177.85.233.250,255.255.255.255,5678,5678,140<134>Sep 28 10:57:43 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,64 ...
nettec@Ubuntu-ELK:/etc/logstash/conf.d$ ls
02-beats-input.conf 10-syslog.conf 10-syslog.conf.save 30-elasticsearch-output.conf
input {
beats {
port => 5044
}
}
udp syslogs tream via 5140
input {
udp {
type => "syslog"
port => 5002
}
}What does your filter and output look like?
filter {
if [type] == "syslog" {
#change to pfSense ip address
if [host] =~ /#\.#\.#\.#/ { #myfirewall Ip
mutate {
add_tag => ["PFSense", "Ready"]
}
}
if "Ready" not in [tags] {
mutate {
add_tag => [ "syslog" ]
}
}
}
}
filter {
if [type] == "syslog" {
mutate {
remove_tag => "Ready"
}
}
}filter {
if [host] =~ /XX\.XX\.XX\.XX/ { #myfirewall Ip
grok {
match => [ "message", "<(?<evtid>.*)>(?<datetime>(?:Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|Jun(?:e)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:tember)?|Oct(?:ober)?|Nov(?:ember)?|Dec(?:ember)?)\s+(?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9]) (?:2[0123]|[01]?[0-9]):(?:[0-5][0-9]):(?:[0-5][0-9])) (?<prog>.*?): (?<msg>.*)" ]
}
mutate {
gsub => ["datetime"," "," "]
}
date {
match => [ "datetime", "MMM dd HH:mm:ss" ]
}
mutate {
replace => [ "message", "%{msg}" ]
}
mutate {
remove_field => [ "msg", "datetime" ]
}
if [prog] =~ /^dhcpd$/ {
mutate {
add_tag => [ "dhcpd" ]
}
}
if [prog] =~ /^suricata/ {
mutate {
add_tag => [ "SuricataIDPS" ]
}
if [message] =~ /^\{/ {
mutate {
gsub => [ "message", "[\u0000]", "\\\"}}" ]
}
json {
source => "message"
}
}
}
if [prog] =~ /^charon$/ {
mutate {
add_tag => [ "ipsec" ]
}
}
if [prog] =~ /^barnyard2/ {
mutate {
add_tag => [ "barnyard2" ]
}
}
if [prog] =~ /^openvpn/ {
mutate {
add_tag => [ "openvpn" ]
}
}
if [prog] =~ /^ntpd/ {
mutate {
add_tag => [ "ntpd" ]
}
}
if [prog] =~ /^php-fpm/ {
mutate {
add_tag => [ "web_portal" ]
}
grok {
patterns_dir => ["/etc/logstash/conf.d/patterns"]
match => [ "message", "%{PFSENSE_APP} %{PFSENSE_APP_DATA}",
"message", "%{PFSENSE_APP} %{PFSENSE_APP_DATA}" ]
}
mutate {
lowercase => [ 'pfsense_ACTION' ]
}
}
if [prog] =~ /^apinger/ {
mutate {
add_tag => [ "apinger" ]
}
}
if [prog] =~ /^filterlog$/ {
mutate {
remove_field => [ "msg", "datetime" ]
}
grok {
add_tag => [ "firewall" ]
patterns_dir => ["/etc/logstash/conf.d/patterns"]
match => [ "message", "%{PFSENSE_LOG_DATA}%{PFSENSE_IP_SPECIFIC_DATA}%{PFSENSE_IP_DATA}%{PFSENSE_PROTOCOL_DATA}",
"message", "%{PFSENSE_LOG_DATA}%{PFSENSE_IPv4_SPECIFIC_DATA_ECN}%{PFSENSE_IP_DATA}%{PFSENSE_PROTOCOL_DATA}" ]
}
mutate {
lowercase => [ 'proto' ]
}
if ![geoip] and [src_ip] !~ /^(10\.|192\.168\.)/ {
geoip {
add_tag => [ "GeoIP" ]
source => "src_ip"
}
}
}
}
}this is the output that I see on kibana
Sep 29 00:20:09 Ubuntu-ELK kibana[19406]: {"type":"response","@timestamp":"2020-09-29T00:20:09Z","tags":,"pid":19406,"method":"get","statusCode":200,"req":{"url":"/ui/fonts/inter_ui/Inter-UI-SemiBold.woff2","method":"get","headers":{"host":"52.199.68.24:5601","connection":"keep-alive","origin":"http://52.199.68.24:5601","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.121 Safari/537.36 Edg/85.0.564.63","accept":"/","referer":"http://52.199.68.24:5601/app/home","accept-encoding":"gzip, deflate","accept-language":"pt-BR,pt;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6"},"remoteAddress":"177.44.12.6","userAgent":"177.44.12.6","referer":"http://52.199.68.24:5601/app/home"},"res":{"statusCode":200,"responseTime":4,"contentLength":9},"message":"GET /ui/fonts/inter_ui/Inter-UI-SemiBold.woff2 200 4ms - 9.0B"}
I have found this filter on the internet because I don't really know how to make a filter. I am an intern and I have to implement this all by myself with very little knowledge
It's a bit confusing reading this entry -
let's sort this out.
What kind of Logs do you want to parse through logstash?
Appache Webserver, Syslogs, Firewall? if so - which firewall.
From there we can try to reconfigure your filter and make this work.
If it's firewall logs - i recommend using kv filter plugin to do the job or use filebeat modules to parse the information.
I need to see PFSENSE firewall logs. If you could show me how to do that, it would be awesome. in the tutorial I used to install ELK I also installed filebeat. Thank you for your help.
Can you give me a censored sample log? so i can check the format - then we can do your Filter.
These are the logs that I need to filter, they are comingo through port 5002, if you could just show me how to display these IPs adress on kibana.
<134>Sep 29 09:03:50 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,128,50714,0,DF,6,tcp,52,10.0.5.27,104.41.13.179,52230,443,0,S,1955399516,,64240,,mss;nop;wscale;nop;nop;sackOK<134>Sep 29 09:03:50 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x10,,64,22944,0,DF,17,udp,76,10.0.5.46,200.189.40.8,54059,123,56<134>Sep 29 09:03:50 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x10,,64,52279,0,DF,17,udp,76,10.0.5.46,200.160.0.8,53139,123,56<134>Sep 29 09:03:51 filterlog: 5,,,1000000103,igb3,match,block,in,4,0x0,,64,0,0,none,17,udp,167,187.62.198.169,255.255.255.255,5678,5678,147<134>Sep 29 09:03:51 filterlog: 5,,,1000000103,igb3,match,block,in,4,0x0,,64,0,0,none,17,udp,142,187.62.195.70,255.255.255.255,5678,5678,122<134>Sep 29 09:03:51 filterlog: 5,,,1000000103,igb3,match,block,in,4,0x0,,64,0,0,none,17,udp,133,187.62.196.137,255.255.255.255,5678,5678,113<134>Sep 29 09:03:52 filterlog: 135,,,1574429241,igb0.10,match,pass,in,4,0x0,,64,0,0,DF,6,tcp,64,172.16.0.43,45.60.201.69,57218,443,0,SEC,1991800226,,65535,,mss;nop;wscale;nop;nop;TS;sackOK;eol<134>Sep 29 09:03:52 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,128,10374,0,DF,6,tcp,52,10.0.5.22,38.113.165.189,51331,443,0,S,3968641758,,8192,,mss;nop;wscale;nop;nop;sackOK<134>Sep 29 09:03:52 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,128,25348,0,none,17,udp,52,10.0.5.25,10.0.5.254,54786,1022,32<134>Sep 29 09:03:52 filterlog: 135,,,1574429241,igb0.10,match,pass,in,4,0x0,,64,0,0,DF,6,tcp,64,172.16.0.43,45.60.201.69,57219,443,0,SEC,2433051567,,65535,,mss;nop;wscale;nop;nop;TS;sackOK;eol<134>Sep 29 09:03:52 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,64,23282,0,DF,6,tcp,60,10.0.5.46,189.125.77.80,44048,80,0,S,2108343532,,14600,,mss;sackOK;TS;nop;wscale<134>Sep 29 09:03:52 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,128,4220,0,DF,6,tcp,52,10.0.5.27,4.28.136.54,52229,80,0,S,1966995828,,64240,,mss;nop;wscale;nop;nop;sackOK<134>Sep 29 09:03:53 filterlog: 5,,,1000000103,igb3,match,block,in,4,0x0,,64,0,0,none,17,udp,167,187.62.196.45,255.255.255.255,5678,5678,147<134>Sep 29 09:03:53 filterlog: 5,,,1000000103,igb3,match,block,in,4,0x0,,64,0,0,none,17,udp,141,187.62.196.122,255.255.255.255,5678,5678,121<134>Sep 29 09:03:53 filterlog: 135,,,1574429241,igb0.10,match,pass,in,4,0x0,,64,0,0,DF,6,tcp,64,172.16.0.43,45.60.201.69,57220,443,0,SEC,1829867287,,65535,,mss;nop;wscale;nop;nop;TS;sackOK;eol<134>Sep 29 09:03:53 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,128,10375,0,DF,6,tcp,52,10.0.5.22,38.113.165.187,51332,443,0,S,2321732742,,8192,,mss;nop;wscale;nop;nop;sackOK<134>Sep 29 09:03:53 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,64,256,0,DF,6,tcp,60,10.0.5.18,189.125.77.80,51050,80,0,S,3293764276,,14600,,mss;sackOK;TS;nop;wscale<134>Sep 29 09:03:53 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,64,23283,0,DF,6,tcp,60,10.0.5.46,189.125.77.80,44048,80,0,S,2108343532,,14600,,mss;sackOK;TS;nop;wscale<134>Sep 29 09:03:53 filterlog: 135,,,1574429241,igb0.10,match,pass,in,4,0x0,,64,0,0,DF,6,tcp,64,172.16.0.43,45.60.201.69,57221,443,0,SEC,55740265,,65535,,mss;nop;wscale;nop;nop;TS;sackOK;eol<134>Sep 29 09:03:53 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,64,48169,0,DF,6,tcp,60,10.0.5.49,189.125.77.80,32893,80,0,S,815336207,,14600,,mss;sackOK;TS;nop;wscale<134>Sep 29 09:03:54 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,128,50715,0,DF,6,tcp,52,10.0.5.27,104.41.13.179,52230,443,0,S,1955399516,,64240,,mss;nop;wscale;nop;nop;sackOK<134>Sep 29 09:03:54 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,128,10377,0,DF,6,tcp,52,10.0.5.22,62.67.238.152,51333,443,0,S,2184934504,,8192,,mss;nop;wscale;nop;nop;sackOK<134>Sep 29 09:03:54 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,64,257,0,DF,6,tcp,60,10.0.5.18,189.125.77.80,51050,80,0,S,3293764276,,14600,,mss;sackOK;TS;nop;wscale<134>Sep 29 09:03:54 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,64,48170,0,DF,6,tcp,60,10.0.5.49,189.125.77.80,32893,80,0,S,815336207,,14600,,mss;sackOK;TS;nop;wscale<134>Sep 29 09:03:56 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,128,10379,0,DF,6,tcp,52,10.0.5.22,38.113.165.187,51332,443,0,S,2321732742,,8192,,mss;nop;wscale;nop;nop;sackOK<134>Sep 29 09:03:56 filterlog: 5,,,1000000103,igb3,match,block,in,4,0x0,,64,0,0,none,17,udp,156,177.85.233.55,255.255.255.255,5678,5678,136<134>Sep 29 09:03:56 filterlog: 135,,,1574429241,igb0.10,match,pass,in,4,0x0,,64,0,0,DF,6,tcp,64,172.16.0.43,45.60.201.69,57222,443,0,SEC,690959946,,65535,,mss;nop;wscale;nop;nop;TS;sackOK;eol<134>Sep 29 09:03:56 filterlog: 136,,,1574429241,igb0.10,match,pass,in,4,0x0,,64,62108,0,none,17,udp,68,172.16.0.43,208.67.222.222,64978,53,48<134>Sep 29 09:03:57 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,64,25236,0,DF,6,tcp,52,10.0.5.230,54.94.16.223,34988,6800,0,S,3477764002,,14600,,mss;nop;nop;sackOK;nop;wscale<134>Sep 29 09:03:57 filterlog: 135,,,1574429241,igb0.10,match,pass,in,4,0x0,,64,0,0,DF,6,tcp,64,172.16.0.43,45.60.201.69,57223,443,0,SEC,4233165489,,65535,,mss;nop;wscale;nop;nop;TS;sackOK;eol<134>Sep 29 09:03:57 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,128,10381,0,DF,6,tcp,52,10.0.5.22,62.67.238.152,51333,443,0,S,2184934504,,8192,,mss;nop;wscale;nop;nop;sackOK<134>Sep 29 09:03:57 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,128,10382,0,DF,6,tcp,52,10.0.5.22,66.110.49.30,51335,443,0,S,305997838,,8192,,mss;nop;wscale;nop;nop;sackOK<134>Sep 29 09:03:57 filterlog: 5,,,1000000103,igb3,match,block,in,4,0x0,,64,0,0,none,17,udp,134,177.85.233.73,255.255.255.255,5678,5678,114<134>Sep 29 09:03:57 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,128,9486,0,DF,6,tcp,52,10.0.5.20,4.28.136.42,21524,80,0,S,509640840,,8192,,mss;nop;wscale;nop;nop;sackOK<134>Sep 29 09:03:57 filterlog: 135,,,1574429241,igb0.10,match,pass,in,4,0x0,,64,0,0,DF,6,tcp,64,172.16.0.43,45.60.201.69,57224,443,0,SEC,1782682395,,65535,,mss;nop;wscale;nop;nop;TS;sackOK;eol<134>Sep 29 09:03:57 filterlog: 5,,,1000000103,igb3,match,block,in,4,0x0,,64,0,0,none,17,udp,163,187.62.195.209,255.255.255.255,5678,5678,143<134>Sep 29 09:03:58 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,128,12420,0,DF,6,tcp,52,10.0.5.27,66.110.49.80,52233,80,0,S,1809942367,,64240,,mss;nop;wscale;nop;nop;sackOK^C
That's hella messi lol.
Can you link me to the page where your filter is from.
Parsing PFSense seems to be an common issue - i just want to see what the guys you followed did.
Actutally the logs is just these
<134>Sep 29 09:13:06 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,64,43993,0,DF,6,tcp,52,10.0.5.230,162.88.193.70,32829,80,0,S,4184300456,,14600,,mss;nop;nop;sackOK;nop;wscale<134>Sep 29 09:13:06
this single line shows that the log entry was triggered by rule id 1000000103 , which resulted in a block action on the igb1 interface. The source and destination IP addresses are shown near the end of the log entry, followed by the source and destination port. Packets from other protocols may show significantly more data.
This is the page I found the filter
PFSense logs have a fixed header followed by a variety of CSV entries. (Format documented here.) I would take off the fixed header using dissect, not grok (the tooling in logstash has improved a lot in the last 6 years)
dissect { mapping => { "message" => "<%{syslog_pri}>%{[@metadata][ts]} %{+[@metadata][ts]} %{+[@metadata][ts]} filterlog: %{restOfLine}" } }
date { match => [ "[@metadata][ts]", "MMM dd HH:mm:ss" ] }
If your dates do not have a year in them then logstash will guess, and sometimes it will guess wrong. See issues 137, 100, and the long discussion of 51.
If you literally just want to extract the IPs then you could do it using grok
grok { match => { "restOfLine" => "%{IP:srcIP},%{IP:dstIP}" } }
If you want to do the job properly, then you need to pull out a set of fields, label them, and determine what the next set of fields are. My initial approach to this was to use dissect again and I got to this point, and realized I had far to go
# dissect will not parse ",," using ",%{fieldName},", so replace it with something we can later delete
mutate { gsub => [ "restOfLine", ",,", ",!!!," ] }
dissect { mapping => { "restOfLine" => "%{rule},%{subRule},%{anchor},%{tracker},%{interface},%{reason},%{action},%{direction},%{ipVersion},%{restOfLine}" } }
if "_dissectfailure" not in [tags] {
if [ipVersion] == "4" {
dissect { mapping => { "restOfLine" => "%{tos},%{ecn},%{ttl},%{id},%{offset},%{flags},%{protocol},%{protocolName},%{restOfLine}" } }
} else if [ipVersion] == "6" {
# Never seen one of thse, "protocol" be the name, in which case protocol and protocolId should change to be like V4
dissect { mapping => { "restOfLine" => "%{class},%{flowLabel},%{hopLimit},%{protocol},%{protocolId},%{restOfLine}" } }
} else {
mutate { add_field => { "parseFailure" => "unknown IP version" } }
}
if [ipVersion] in [ "4", "6" ] {
dissect { mapping => { "restOfLine" => "%{length},%{srcIp},%{dstIp},%{restOfLine}" } }
if [protocol] in [ "6", "17" ] {
# TCP and UDP
dissect { mapping => { "restOfLine" => "%{srcPort},%{dstPort},%{dataLength},%{restOfLine}" } }
} else if [protocol] == "112" {
# CARP
dissect { mapping => { "restOfLine" => "%{type},%{carpTtl},%{vhid},%{carpVersion},%{advskew},%{advbase},%{restOfLine}" } }
}
if [protocol] == "6" {
dissect { mapping => { "restOfLine" => "%{tcpFlags},%{tcpSeq},%{tcpAck},%{tcpWindow},%{tcpUrg},%{tcpOptions}" } }
} else if [protocol] == "1" {
# ICMP
dissect { mapping => { "restOfLine" => "%{icmpType}},%{restOfLine}" } }
# Lots more code to dissect each of these ...
}
}
}
Note that that code is incomplete and fails to parse UDP correctly because 'if [protocol] in [ "6", "17" ] {' needs to be two separate branches...
I came up with a different solution... Create a ruby script file that contains
def register(params)
@keys = params['keys']
@values = params['values']
end
def filter(event)
values = event.get(@values)
@keys.each { |k|
event.set(k, values.shift)
}
event.set(@values, values)
[event]
end
You call that using a ruby filter like
ruby {
path => "/home/user/addFields.rb"
script_params => {
"values" => "restOfLine"
"keys" => [ "tos", "ecn", "ttl", "id", "offset", "flags", "protocol", "protocolName" ]
}
}
That assumes that restOfLine is an array that contains values. It will remove entries from the array for each of the keys and add them to the event.
Using that like this
dissect { mapping => { "message" => "<%{syslog_pri}>%{[@metadata][ts]} %{+[@metadata][ts]} %{+[@metadata][ts]} filterlog: %{restOfLine}" } }
date { match => [ "[@metadata][ts]", "MMM dd HH:mm:ss" ] }
mutate { split => { "restOfLine" => "," } }
ruby {
path => "/home/user/addFields.rb"
script_params => {
"values" => "restOfLine"
"keys" => [ "rule", "subrule", "anchor", "tracker", "interface", "reason", "action", "direction", "ipVersion" ]
}
}
if [ipVersion] == "4" {
ruby {
path => "/home/user/addFields.rb"
script_params => {
"values" => "restOfLine"
"keys" => [ "tos", "ecn", "ttl", "id", "offset", "flags", "protocol", "protocolName" ]
}
}
} else if [ipVersion] == "6" {
# Never seen one of these, "protocol" could be the name, in which case protocol and protocolId should change to be like V4
ruby {
path => "/home/user/addFields.rb"
script_params => {
"values" => "restOfLine"
"keys" => [ "class", "flowLabel", "hopLimit", "protocol", "protocolId" ]
}
}
} else {
mutate { add_field => { "parseFailure" => "unknown IP version" } }
}
if [ipVersion] in [ "4", "6" ] {
ruby {
path => "/home/user/addFields.rb"
script_params => {
"values" => "restOfLine"
"keys" => [ "length", "srcIp", "dstIp" ]
}
}
if [protocol] == "1" {
# ICMP
ruby {
path => "/home/user/addFields.rb"
script_params => {
"values" => "restOfLine"
"keys" => [ "icmpType" ]
}
}
if [icmpType] == "0" {
# Echo reply
# I am not going to write all these!
} else if [icmpType] == "3" {
# Destination unreachable
# Need to parse the codes etc
}
# ... and the other types
} else if [protocol] == "6" {
# TCP
ruby {
path => "/home/user/addFields.rb"
script_params => {
"values" => "restOfLine"
"keys" => [ "srcPort", "dstPort", "dataLength", "tcpFlags", "tcpSeq", "tcpAck", "tcpWindow", "tcpUrg", "tcpOptions" ]
}
}
} else if [protocol] == "17" {
# UDP
ruby {
path => "/home/user/addFields.rb"
script_params => {
"values" => "restOfLine"
"keys" => [ "srcPort", "dstPort", "dataLength" ]
}
}
} else if [protocol] == "112" {
# CARP
ruby {
path => "/home/user/addFields.rb"
script_params => {
"values" => "restOfLine"
"keys" => [ "type", "carpTtl", "vhid", "carpVersion", "advskew", "advbase" ]
}
}
}
}
}
Will get you udp events like
{
"anchor" => "",
"interface" => "igb3",
"tos" => "0x0",
"rule" => "5",
"protocol" => "17",
"protocolName" => "udp",
"dstPort" => "5678",
"tracker" => "1000000103",
"id" => "0",
"srcPort" => "5678",
"srcIp" => "177.85.233.73",
"dstIp" => "255.255.255.255",
"ttl" => "64",
"offset" => "0",
"direction" => "in",
"reason" => "match",
"flags" => "none",
"subrule" => "",
"message" => "<134>Sep 29 09:03:57 filterlog: 5,,,1000000103,igb3,match,block,in,4,0x0,,64,0,0,none,17,udp,134,177.85.233.73,255.255.255.255,5678,5678,114",
"length" => "134",
"@version" => "1",
"ecn" => "",
"@timestamp" => 2020-09-29T13:03:57.000Z,
"syslog_pri" => "134",
"ipVersion" => "4",
"action" => "block",
"restOfLine" => [],
"dataLength" => "114"
}
and for tcp events
{
"anchor" => "",
"interface" => "igb1",
"tcpSeq" => "2184934504",
"tos" => "0x0",
"rule" => "5",
"protocol" => "6",
"protocolName" => "tcp",
"dstPort" => "443",
"tracker" => "1000000103",
"id" => "10381",
"srcPort" => "51333",
"srcIp" => "10.0.5.22",
"dstIp" => "62.67.238.152",
"tcpAck" => "",
"ttl" => "128",
"tcpWindow" => "8192",
"tcpOptions" => "mss;nop;wscale;nop;nop;sackOK",
"offset" => "0",
"direction" => "in",
"reason" => "match",
"flags" => "DF",
"subrule" => "",
"message" => "<134>Sep 29 09:03:57 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,128,10381,0,DF,6,tcp,52,10.0.5.22,62.67.238.152,51333,443,0,S,2184934504,,8192,,mss;nop;wscale;nop;nop;sackOK",
"length" => "52",
"tcpUrg" => "",
"@version" => "1",
"ecn" => "",
"@timestamp" => 2020-09-29T13:03:57.000Z,
"syslog_pri" => "134",
"ipVersion" => "4",
"action" => "block",
"restOfLine" => [],
"dataLength" => "0",
"tcpFlags" => "S"
}
I leave it as an exercise for the reader to add IPV6 handling, ICMP handling, and error handling.
3 Likes
Thank you so much, I am going to try that now and I will post the results afterwards
This is my configuration now, but I still can't see any IPs. I do want to do the job properly, but firset I would like to see it working by showing the IPs on kibana. Is there something wrong with this?
input {
udp {
type => "syslog"
port => 5002
}
}
filter{
grok { match => { "restOfLine" => "%{IP:srcIP},%{IP:dstIP}" } }
}You have not used dissect to create the restOfLine field. You could use grok to match against the [message] field.
still, completly unresponsive, I think there might be something wrong with some configuration. I can see every logs from my server, but when it comes to the remote pfsense firewall, nothing