Can anyone please help me how can i parse this firewall log?

Elastic Stack Logstash
1

date stamp timestamp(data)private IP [public IP]->[public IP(client side)(protocol)

->i am not able to understand the type of log and i am new in the field so kindly please help me to parse the data of this kind of log

2

can you please past one line so we can able to fit the grok.

in the mean time try to use this

3

yes you can use this as a sample log :
2000-06-30 01:44:00: {exwire-NAT-Rules} PSERVE_SESSION_OPEN: application:none, lsi.32 101.22.954.358.84355 [115.109.546.105.62589] -> 66.451.684.74:99 (TCP)

4

I would start with

grok { match => { "message" => "%{DATA:[@metadata][startOfLine]} %{IPV4:ip1}.%{INT:port1} \[%{IPV4:ip2}.%{INT:port2}\] -> %{IPV4:ip3}:%{INT:port3} \(%{WORD:protocol}\)$" } }
dissect { mapping => { "[@metadata][startOfLine]" => "%{[@metadata][ts]} %{+[@metadata][ts]}: %{data}" } }
date { match => [ "[@metadata][ts]", "YYYY-MM-dd HH:mm:ss" ] }
5

The filter you sent me above isn't working correctly and i am not able to find the problem .Can you please explain what could be the possible reason for not expected answer.
the error coming out is that the:-
Provided Grok patterns do not match data in the input .

6

Have you tested it against an actual log message rather than the example you gave? The IP addresses in the example are not valid, so IPV4 does not match them.

7

Yes , i have tried them on actual logs. but they doesn't work.

8

They work for me if I change your IP addresses and port numbers to be valid

input { generator { count => 1 lines => [
'2000-06-30 01:44:00: {exwire-NAT-Rules} PSERVE_SESSION_OPEN: application:none, lsi.32 101.22.54.58.4355 [115.109.46.105.62589] -> 66.51.84.74:99 (TCP)',
'2000-06-30 01:44:00: {exwire-NAT-Rules} PSERVE_SESSION_OPEN: application:none, lsi.32 101.22.95.35.8435 [115.109.56.105.6589] -> 66.41.64.74:99 (TCP)' ] } }

filter {
    grok { match => { "message" => "%{DATA:[@metadata][startOfLine]} %{IPV4:ip1}.%{INT:port1} \[%{IPV4:ip2}.%{INT:port2}\] -> %{IPV4:ip3}:%{INT:port3} \(%{WORD:protocol}\)$" } }
    dissect { mapping => { "[@metadata][startOfLine]" => "%{[@metadata][ts]} %{+[@metadata][ts]}: %{data}" } }
    date { match => [ "[@metadata][ts]", "YYYY-MM-dd HH:mm:ss" ] }
}
output { stdout { codec => rubydebug { metadata => false } } }

gets me

{
"ip1" => "101.22.95.35",
"ip2" => "115.109.56.105",
"port2" => "6589",
"data" => "{exwire-NAT-Rules} PSERVE_SESSION_OPEN: application:none, lsi.32",
"protocol" => "TCP",

etc.

9

Thank you ,it is working now . But i want to actually use these logs in kibana but i am unable to connect to the elasticsearch
TAKE A LOOK AT THE CONFIGURATION FILE I MADE FOR THE SAME ( i just added yo your file some output commands) :-

output {

     elasticsearch { 
                 hosts => ["localhost:9200"]
                 user => "5d5a"
                 password => "33f7"
                 index => "syslog-%{+YYYY.MM.dd}"
                 document_type => "custom_logs"
                 }
   }
}

(and as i am doing it for small log file i am manually inputting it through the log file and not using beats)

10

"document_type" was deprecated from Elasticsearch 6.0+. Remove it and see If that works.

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.