# Configure logstash to show logs from port 5002

**URL:** <https://discuss.elastic.co/t/configure-logstash-to-show-logs-from-port-5002/250205>\
**Category:** Logstash\
**Created:** [September 28, 2020, 2:08pm UTC](https://discuss.elastic.co/t/configure-logstash-to-show-logs-from-port-5002/250205 "2020-09-28T14:08:23Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Lusca\_lusca](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lusca_lusca/32/45091_2.png) [@Lusca\_lusca](https://discuss.elastic.co/u/Lusca_lusca)\
**Post date:** [September 28, 2020, 2:08pm UTC](https://discuss.elastic.co/t/configure-logstash-to-show-logs-from-port-5002/250205/1 "2020-09-28T14:08:24Z")

</div>

I am knew to ELK and I can't see the logs coming from port 5002 on kibana. Can anyone please help me configure logstash to send the logs to kibana?

---

<div class="post-metadata">

**Author:** ![Lusca\_lusca](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lusca_lusca/32/45091_2.png) [@Lusca\_lusca](https://discuss.elastic.co/u/Lusca_lusca)\
**Post date:** [September 28, 2020, 2:09pm UTC](https://discuss.elastic.co/t/configure-logstash-to-show-logs-from-port-5002/250205/2 "2020-09-28T14:09:57Z")

</div>

I know the logs are coming:

nettec@Ubuntu-ELK:/etc/logstash/conf.d$ nc -l -u 10.0.1.4 5002

\<134\>Sep 28 10:57:43 filterlog: 5,,,1000000103,igb3,match,block,in,4,0x0,,64,0,0,none,17,udp,149,177.85.233.177,255.255.255.255,5678,5678,129\<134\>Sep 28 10:57:43 filterlog: 5,,,1000000103,igb3,match,block,in,4,0x0,,64,0,0,none,17,udp,160,177.85.233.250,255.255.255.255,5678,5678,140\<134\>Sep 28 10:57:43 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,64 ...

---

<div class="post-metadata">

**Author:** ![Lusca\_lusca](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lusca_lusca/32/45091_2.png) [@Lusca\_lusca](https://discuss.elastic.co/u/Lusca_lusca)\
**Post date:** [September 28, 2020, 2:15pm UTC](https://discuss.elastic.co/t/configure-logstash-to-show-logs-from-port-5002/250205/5 "2020-09-28T14:15:55Z")

</div>

nettec@Ubuntu-ELK:/etc/logstash/conf.d$ ls  
02-beats-input.conf 10-syslog.conf 10-syslog.conf.save 30-elasticsearch-output.conf

```
input {
  beats {
    port => 5044
  }
}

udp syslogs tream via 5140
input {
  udp {
    type => "syslog"
    port => 5002
  }
}
```

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [September 28, 2020, 11:22pm UTC](https://discuss.elastic.co/t/configure-logstash-to-show-logs-from-port-5002/250205/6 "2020-09-28T23:22:56Z")

</div>

What does your filter and output look like?

---

<div class="post-metadata">

**Author:** ![Lusca\_lusca](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lusca_lusca/32/45091_2.png) [@Lusca\_lusca](https://discuss.elastic.co/u/Lusca_lusca)\
**Post date:** [September 29, 2020, 12:13am UTC](https://discuss.elastic.co/t/configure-logstash-to-show-logs-from-port-5002/250205/7 "2020-09-29T00:13:53Z")

</div>

```
filter {
    if [type] == "syslog" {

    #change to pfSense ip address
    if [host] =~ /#\.#\.#\.#/ { #myfirewall Ip
      mutate {
        add_tag => ["PFSense", "Ready"]
      }
    }

    if "Ready" not in [tags] {
      mutate {
        add_tag => ["syslog"]
      }
    }
  }
}

filter {
  if [type] == "syslog" {
    mutate {
      remove_tag => "Ready"
    }
  }
}
```

---

<div class="post-metadata">

**Author:** ![Lusca\_lusca](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lusca_lusca/32/45091_2.png) [@Lusca\_lusca](https://discuss.elastic.co/u/Lusca_lusca)\
**Post date:** [September 29, 2020, 12:15am UTC](https://discuss.elastic.co/t/configure-logstash-to-show-logs-from-port-5002/250205/8 "2020-09-29T00:15:16Z")

</div>

```
filter {
	if [host] =~ /XX\.XX\.XX\.XX/ { #myfirewall Ip
	    grok {
	      match => ["message", "<(?<evtid>.*)>(?<datetime>(?:Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|Jun(?:e)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:tember)?|Oct(?:ober)?|Nov(?:ember)?|Dec(?:ember)?)\s+(?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9]) (?:2[0123]|[01]?[0-9]):(?:[0-5][0-9]):(?:[0-5][0-9])) (?<prog>.*?): (?<msg>.*)" ]
	    }
	    mutate {
	      gsub => ["datetime"," "," "]
	    }
	    date {
	      match => ["datetime", "MMM dd HH:mm:ss"]
	    }
	    mutate {
	      replace => ["message", "%{msg}"]
	    }
	    mutate {
	      remove_field => ["msg", "datetime"]
	    }
            if [prog] =~ /^dhcpd$/ {
              mutate {
		add_tag => ["dhcpd"]
              }
            }
            if [prog] =~ /^suricata/ {
              mutate {
		add_tag => ["SuricataIDPS"]
              }
	      if [message] =~ /^\{/ {
		      mutate {
			gsub => ["message", "[\u0000]", "\\\"}}" ]
		      }
		      json {
			source => "message"
		      }
	      }
            }
            if [prog] =~ /^charon$/ {
              mutate {
                add_tag => ["ipsec"]
              }
            }
            if [prog] =~ /^barnyard2/ {
              mutate {
                add_tag => ["barnyard2"]
              }
            }
            if [prog] =~ /^openvpn/ {
              mutate {
                add_tag => ["openvpn"]
              }
            }
            if [prog] =~ /^ntpd/ {
              mutate {
                add_tag => ["ntpd"]
              }
            }
            if [prog] =~ /^php-fpm/ {
              mutate {
                add_tag => ["web_portal"]
              }
              grok {
                  patterns_dir => ["/etc/logstash/conf.d/patterns"]
                  match => [ "message", "%{PFSENSE_APP} %{PFSENSE_APP_DATA}",
                  "message", "%{PFSENSE_APP} %{PFSENSE_APP_DATA}" ]
              }
              mutate {
                  lowercase => ['pfsense_ACTION']
              }
            }
            if [prog] =~ /^apinger/ {
              mutate {
                add_tag => ["apinger"]
              }
            }
	    if [prog] =~ /^filterlog$/ {
	        mutate {
                    remove_field => ["msg", "datetime"]
	        }
                grok {
                    add_tag => ["firewall"]
                    patterns_dir => ["/etc/logstash/conf.d/patterns"]
                    match => [ "message", "%{PFSENSE_LOG_DATA}%{PFSENSE_IP_SPECIFIC_DATA}%{PFSENSE_IP_DATA}%{PFSENSE_PROTOCOL_DATA}",
                    "message", "%{PFSENSE_LOG_DATA}%{PFSENSE_IPv4_SPECIFIC_DATA_ECN}%{PFSENSE_IP_DATA}%{PFSENSE_PROTOCOL_DATA}" ]
                }
                mutate {
                    lowercase => ['proto']
                }
		if ![geoip] and [src_ip] !~ /^(10\.|192\.168\.)/ {
                  geoip {
                    add_tag => ["GeoIP"]
                    source => "src_ip"
                  }
		}
            }
	}
}
```

---

<div class="post-metadata">

**Author:** ![Lusca\_lusca](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lusca_lusca/32/45091_2.png) [@Lusca\_lusca](https://discuss.elastic.co/u/Lusca_lusca)\
**Post date:** [September 29, 2020, 12:22am UTC](https://discuss.elastic.co/t/configure-logstash-to-show-logs-from-port-5002/250205/9 "2020-09-29T00:22:42Z")

</div>

this is the output that I see on kibana

Sep 29 00:20:09 Ubuntu-ELK kibana[19406]: {"type":"response","@timestamp":"2020-09-29T00:20:09Z","tags":,"pid":19406,"method":"get","statusCode":200,"req":{"url":"/ui/fonts/inter\_ui/Inter-UI-SemiBold.woff2","method":"get","headers":{"host":"52.199.68.24:5601","connection":"keep-alive","origin":"[http://52.199.68.24:5601](http://52.199.68.24:5601)","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.121 Safari/537.36 Edg/85.0.564.63","accept":"_/_","referer":"[http://52.199.68.24:5601/app/home","accept-encoding":"gzip](http://52.199.68.24:5601/app/home%22,%22accept-encoding%22:%22gzip), deflate","accept-language":"pt-BR,pt;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6"},"remoteAddress":"177.44.12.6","userAgent":"177.44.12.6","referer":"[http://52.199.68.24:5601/app/home"},"res":{"statusCode":200,"responseTime":4,"contentLength":9},"message":"GET](http://52.199.68.24:5601/app/home%22%7D,%22res%22:%7B%22statusCode%22:200,%22responseTime%22:4,%22contentLength%22:9%7D,%22message%22:%22GET) /ui/fonts/inter\_ui/Inter-UI-SemiBold.woff2 200 4ms - 9.0B"}

---

<div class="post-metadata">

**Author:** ![Lusca\_lusca](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lusca_lusca/32/45091_2.png) [@Lusca\_lusca](https://discuss.elastic.co/u/Lusca_lusca)\
**Post date:** [September 29, 2020, 12:24am UTC](https://discuss.elastic.co/t/configure-logstash-to-show-logs-from-port-5002/250205/10 "2020-09-29T00:24:35Z")

</div>

I have found this filter on the internet because I don't really know how to make a filter. I am an intern and I have to implement this all by myself with very little knowledge

---

<div class="post-metadata">

**Author:** ![Moritz\_Kiesewetter](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/moritz_kiesewetter/32/51243_2.png) [@Moritz\_Kiesewetter](https://discuss.elastic.co/u/Moritz_Kiesewetter)\
**Post date:** [September 29, 2020, 10:41am UTC](https://discuss.elastic.co/t/configure-logstash-to-show-logs-from-port-5002/250205/11 "2020-09-29T10:41:32Z")

</div>

It's a bit confusing reading this entry -  
let's sort this out.

What kind of Logs do you want to parse through logstash?  
Appache Webserver, Syslogs, Firewall? if so - which firewall.  
From there we can try to reconfigure your filter and make this work.

If it's firewall logs - i recommend using kv filter plugin to do the job or use filebeat modules to parse the information.

---

<div class="post-metadata">

**Author:** ![Lusca\_lusca](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lusca_lusca/32/45091_2.png) [@Lusca\_lusca](https://discuss.elastic.co/u/Lusca_lusca)\
**Post date:** [September 29, 2020, 11:03am UTC](https://discuss.elastic.co/t/configure-logstash-to-show-logs-from-port-5002/250205/12 "2020-09-29T11:03:25Z")

</div>

I need to see PFSENSE firewall logs. If you could show me how to do that, it would be awesome. in the tutorial I used to install ELK I also installed filebeat. Thank you for your help.

---

<div class="post-metadata">

**Author:** ![Moritz\_Kiesewetter](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/moritz_kiesewetter/32/51243_2.png) [@Moritz\_Kiesewetter](https://discuss.elastic.co/u/Moritz_Kiesewetter)\
**Post date:** [September 29, 2020, 11:19am UTC](https://discuss.elastic.co/t/configure-logstash-to-show-logs-from-port-5002/250205/13 "2020-09-29T11:19:51Z")

</div>

Can you give me a censored sample log? so i can check the format - then we can do your Filter.

---

<div class="post-metadata">

**Author:** ![Lusca\_lusca](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lusca_lusca/32/45091_2.png) [@Lusca\_lusca](https://discuss.elastic.co/u/Lusca_lusca)\
**Post date:** [September 29, 2020, 12:05pm UTC](https://discuss.elastic.co/t/configure-logstash-to-show-logs-from-port-5002/250205/14 "2020-09-29T12:05:53Z")

</div>

These are the logs that I need to filter, they are comingo through port 5002, if you could just show me how to display these IPs adress on kibana.

\<134\>Sep 29 09:03:50 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,128,50714,0,DF,6,tcp,52,10.0.5.27,104.41.13.179,52230,443,0,S,1955399516,,64240,,mss;nop;wscale;nop;nop;sackOK\<134\>Sep 29 09:03:50 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x10,,64,22944,0,DF,17,udp,76,10.0.5.46,200.189.40.8,54059,123,56\<134\>Sep 29 09:03:50 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x10,,64,52279,0,DF,17,udp,76,10.0.5.46,200.160.0.8,53139,123,56\<134\>Sep 29 09:03:51 filterlog: 5,,,1000000103,igb3,match,block,in,4,0x0,,64,0,0,none,17,udp,167,187.62.198.169,255.255.255.255,5678,5678,147\<134\>Sep 29 09:03:51 filterlog: 5,,,1000000103,igb3,match,block,in,4,0x0,,64,0,0,none,17,udp,142,187.62.195.70,255.255.255.255,5678,5678,122\<134\>Sep 29 09:03:51 filterlog: 5,,,1000000103,igb3,match,block,in,4,0x0,,64,0,0,none,17,udp,133,187.62.196.137,255.255.255.255,5678,5678,113\<134\>Sep 29 09:03:52 filterlog: 135,,,1574429241,igb0.10,match,pass,in,4,0x0,,64,0,0,DF,6,tcp,64,172.16.0.43,45.60.201.69,57218,443,0,SEC,1991800226,,65535,,mss;nop;wscale;nop;nop;TS;sackOK;eol\<134\>Sep 29 09:03:52 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,128,10374,0,DF,6,tcp,52,10.0.5.22,38.113.165.189,51331,443,0,S,3968641758,,8192,,mss;nop;wscale;nop;nop;sackOK\<134\>Sep 29 09:03:52 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,128,25348,0,none,17,udp,52,10.0.5.25,10.0.5.254,54786,1022,32\<134\>Sep 29 09:03:52 filterlog: 135,,,1574429241,igb0.10,match,pass,in,4,0x0,,64,0,0,DF,6,tcp,64,172.16.0.43,45.60.201.69,57219,443,0,SEC,2433051567,,65535,,mss;nop;wscale;nop;nop;TS;sackOK;eol\<134\>Sep 29 09:03:52 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,64,23282,0,DF,6,tcp,60,10.0.5.46,189.125.77.80,44048,80,0,S,2108343532,,14600,,mss;sackOK;TS;nop;wscale\<134\>Sep 29 09:03:52 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,128,4220,0,DF,6,tcp,52,10.0.5.27,4.28.136.54,52229,80,0,S,1966995828,,64240,,mss;nop;wscale;nop;nop;sackOK\<134\>Sep 29 09:03:53 filterlog: 5,,,1000000103,igb3,match,block,in,4,0x0,,64,0,0,none,17,udp,167,187.62.196.45,255.255.255.255,5678,5678,147\<134\>Sep 29 09:03:53 filterlog: 5,,,1000000103,igb3,match,block,in,4,0x0,,64,0,0,none,17,udp,141,187.62.196.122,255.255.255.255,5678,5678,121\<134\>Sep 29 09:03:53 filterlog: 135,,,1574429241,igb0.10,match,pass,in,4,0x0,,64,0,0,DF,6,tcp,64,172.16.0.43,45.60.201.69,57220,443,0,SEC,1829867287,,65535,,mss;nop;wscale;nop;nop;TS;sackOK;eol\<134\>Sep 29 09:03:53 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,128,10375,0,DF,6,tcp,52,10.0.5.22,38.113.165.187,51332,443,0,S,2321732742,,8192,,mss;nop;wscale;nop;nop;sackOK\<134\>Sep 29 09:03:53 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,64,256,0,DF,6,tcp,60,10.0.5.18,189.125.77.80,51050,80,0,S,3293764276,,14600,,mss;sackOK;TS;nop;wscale\<134\>Sep 29 09:03:53 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,64,23283,0,DF,6,tcp,60,10.0.5.46,189.125.77.80,44048,80,0,S,2108343532,,14600,,mss;sackOK;TS;nop;wscale\<134\>Sep 29 09:03:53 filterlog: 135,,,1574429241,igb0.10,match,pass,in,4,0x0,,64,0,0,DF,6,tcp,64,172.16.0.43,45.60.201.69,57221,443,0,SEC,55740265,,65535,,mss;nop;wscale;nop;nop;TS;sackOK;eol\<134\>Sep 29 09:03:53 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,64,48169,0,DF,6,tcp,60,10.0.5.49,189.125.77.80,32893,80,0,S,815336207,,14600,,mss;sackOK;TS;nop;wscale\<134\>Sep 29 09:03:54 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,128,50715,0,DF,6,tcp,52,10.0.5.27,104.41.13.179,52230,443,0,S,1955399516,,64240,,mss;nop;wscale;nop;nop;sackOK\<134\>Sep 29 09:03:54 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,128,10377,0,DF,6,tcp,52,10.0.5.22,62.67.238.152,51333,443,0,S,2184934504,,8192,,mss;nop;wscale;nop;nop;sackOK\<134\>Sep 29 09:03:54 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,64,257,0,DF,6,tcp,60,10.0.5.18,189.125.77.80,51050,80,0,S,3293764276,,14600,,mss;sackOK;TS;nop;wscale\<134\>Sep 29 09:03:54 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,64,48170,0,DF,6,tcp,60,10.0.5.49,189.125.77.80,32893,80,0,S,815336207,,14600,,mss;sackOK;TS;nop;wscale\<134\>Sep 29 09:03:56 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,128,10379,0,DF,6,tcp,52,10.0.5.22,38.113.165.187,51332,443,0,S,2321732742,,8192,,mss;nop;wscale;nop;nop;sackOK\<134\>Sep 29 09:03:56 filterlog: 5,,,1000000103,igb3,match,block,in,4,0x0,,64,0,0,none,17,udp,156,177.85.233.55,255.255.255.255,5678,5678,136\<134\>Sep 29 09:03:56 filterlog: 135,,,1574429241,igb0.10,match,pass,in,4,0x0,,64,0,0,DF,6,tcp,64,172.16.0.43,45.60.201.69,57222,443,0,SEC,690959946,,65535,,mss;nop;wscale;nop;nop;TS;sackOK;eol\<134\>Sep 29 09:03:56 filterlog: 136,,,1574429241,igb0.10,match,pass,in,4,0x0,,64,62108,0,none,17,udp,68,172.16.0.43,208.67.222.222,64978,53,48\<134\>Sep 29 09:03:57 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,64,25236,0,DF,6,tcp,52,10.0.5.230,54.94.16.223,34988,6800,0,S,3477764002,,14600,,mss;nop;nop;sackOK;nop;wscale\<134\>Sep 29 09:03:57 filterlog: 135,,,1574429241,igb0.10,match,pass,in,4,0x0,,64,0,0,DF,6,tcp,64,172.16.0.43,45.60.201.69,57223,443,0,SEC,4233165489,,65535,,mss;nop;wscale;nop;nop;TS;sackOK;eol\<134\>Sep 29 09:03:57 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,128,10381,0,DF,6,tcp,52,10.0.5.22,62.67.238.152,51333,443,0,S,2184934504,,8192,,mss;nop;wscale;nop;nop;sackOK\<134\>Sep 29 09:03:57 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,128,10382,0,DF,6,tcp,52,10.0.5.22,66.110.49.30,51335,443,0,S,305997838,,8192,,mss;nop;wscale;nop;nop;sackOK\<134\>Sep 29 09:03:57 filterlog: 5,,,1000000103,igb3,match,block,in,4,0x0,,64,0,0,none,17,udp,134,177.85.233.73,255.255.255.255,5678,5678,114\<134\>Sep 29 09:03:57 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,128,9486,0,DF,6,tcp,52,10.0.5.20,4.28.136.42,21524,80,0,S,509640840,,8192,,mss;nop;wscale;nop;nop;sackOK\<134\>Sep 29 09:03:57 filterlog: 135,,,1574429241,igb0.10,match,pass,in,4,0x0,,64,0,0,DF,6,tcp,64,172.16.0.43,45.60.201.69,57224,443,0,SEC,1782682395,,65535,,mss;nop;wscale;nop;nop;TS;sackOK;eol\<134\>Sep 29 09:03:57 filterlog: 5,,,1000000103,igb3,match,block,in,4,0x0,,64,0,0,none,17,udp,163,187.62.195.209,255.255.255.255,5678,5678,143\<134\>Sep 29 09:03:58 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,128,12420,0,DF,6,tcp,52,10.0.5.27,66.110.49.80,52233,80,0,S,1809942367,,64240,,mss;nop;wscale;nop;nop;sackOK^C

---

<div class="post-metadata">

**Author:** ![Moritz\_Kiesewetter](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/moritz_kiesewetter/32/51243_2.png) [@Moritz\_Kiesewetter](https://discuss.elastic.co/u/Moritz_Kiesewetter)\
**Post date:** [September 29, 2020, 12:27pm UTC](https://discuss.elastic.co/t/configure-logstash-to-show-logs-from-port-5002/250205/15 "2020-09-29T12:27:57Z")

</div>

That's hella messi lol.  
Can you link me to the page where your filter is from.

Parsing PFSense seems to be an common issue - i just want to see what the guys you followed did.

---

<div class="post-metadata">

**Author:** ![Lusca\_lusca](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lusca_lusca/32/45091_2.png) [@Lusca\_lusca](https://discuss.elastic.co/u/Lusca_lusca)\
**Post date:** [September 29, 2020, 12:42pm UTC](https://discuss.elastic.co/t/configure-logstash-to-show-logs-from-port-5002/250205/16 "2020-09-29T12:42:54Z")

</div>

Actutally the logs is just these

\<134\>Sep 29 09:13:06 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,64,43993,0,DF,6,tcp,52,10.0.5.230,162.88.193.70,32829,80,0,S,4184300456,,14600,,mss;nop;nop;sackOK;nop;wscale\<134\>Sep 29 09:13:06

this single line shows that the log entry was triggered by rule id `1000000103` , which resulted in a block action on the `igb1` interface. The source and destination IP addresses are shown near the end of the log entry, followed by the source and destination port. Packets from other protocols may show significantly more data.

---

<div class="post-metadata">

**Author:** ![Lusca\_lusca](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lusca_lusca/32/45091_2.png) [@Lusca\_lusca](https://discuss.elastic.co/u/Lusca_lusca)\
**Post date:** [September 29, 2020, 12:44pm UTC](https://discuss.elastic.co/t/configure-logstash-to-show-logs-from-port-5002/250205/17 "2020-09-29T12:44:29Z")

</div>

This is the page I found the filter

> **[Updated: Monitoring pfSense (2.1 & 2.2) logs using ELK (ElasticSearch,...](https://elijahpaul.co.uk/monitoring-pfsense-2-1-logs-using-elk-logstash-kibana-elasticsearch/)**
>
> Scroll to the bottom for the update on applying this tutorial to the new pfSense 2.2 log format What is pfSense? Only the best open source, software based firewall there is (I'm biased). I use it a lot, especially in virtualized environments....

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [September 29, 2020, 5:33pm UTC](https://discuss.elastic.co/t/configure-logstash-to-show-logs-from-port-5002/250205/18 "2020-09-29T17:33:26Z")

</div>

PFSense logs have a fixed header followed by a variety of CSV entries. (Format documented [here](https://docs.netgate.com/pfsense/en/latest/monitoring/logs/raw-filter-format.html).) I would take off the fixed header using dissect, not grok (the tooling in logstash has improved a lot in the last 6 years)

```
    dissect { mapping => { "message" => "<%{syslog_pri}>%{[@metadata][ts]} %{+[@metadata][ts]} %{+[@metadata][ts]} filterlog: %{restOfLine}" } }
    date { match => ["[@metadata][ts]", "MMM dd HH:mm:ss" ] }

```

If your dates do not have a year in them then logstash will guess, and sometimes it will guess wrong. See issues [137](https://github.com/logstash-plugins/logstash-filter-date/issues/137), [100](https://github.com/logstash-plugins/logstash-filter-date/issues/100), and the long discussion of [51](https://github.com/logstash-plugins/logstash-filter-date/issues/51).

If you literally just want to extract the IPs then you could do it using grok

```
    grok { match => { "restOfLine" => "%{IP:srcIP},%{IP:dstIP}" } }

```

If you want to do the job properly, then you need to pull out a set of fields, label them, and determine what the next set of fields are. My initial approach to this was to use dissect again and I got to this point, and realized I had far to go

```
    # dissect will not parse ",," using ",%{fieldName},", so replace it with something we can later delete
    mutate { gsub => ["restOfLine", ",,", ",!!!,"] }

    dissect { mapping => { "restOfLine" => "%{rule},%{subRule},%{anchor},%{tracker},%{interface},%{reason},%{action},%{direction},%{ipVersion},%{restOfLine}" } }
    if "_dissectfailure" not in [tags] {
        if [ipVersion] == "4" {
            dissect { mapping => { "restOfLine" => "%{tos},%{ecn},%{ttl},%{id},%{offset},%{flags},%{protocol},%{protocolName},%{restOfLine}" } }
        } else if [ipVersion] == "6" {
            # Never seen one of thse, "protocol" be the name, in which case protocol and protocolId should change to be like V4
            dissect { mapping => { "restOfLine" => "%{class},%{flowLabel},%{hopLimit},%{protocol},%{protocolId},%{restOfLine}" } }
        } else {
            mutate { add_field => { "parseFailure" => "unknown IP version" } }
        }

        if [ipVersion] in ["4", "6"] {
            dissect { mapping => { "restOfLine" => "%{length},%{srcIp},%{dstIp},%{restOfLine}" } }
            if [protocol] in ["6", "17"] {
                # TCP and UDP
                dissect { mapping => { "restOfLine" => "%{srcPort},%{dstPort},%{dataLength},%{restOfLine}" } }
            } else if [protocol] == "112" {
                # CARP
                dissect { mapping => { "restOfLine" => "%{type},%{carpTtl},%{vhid},%{carpVersion},%{advskew},%{advbase},%{restOfLine}" } }
            }
            if [protocol] == "6" {
                dissect { mapping => { "restOfLine" => "%{tcpFlags},%{tcpSeq},%{tcpAck},%{tcpWindow},%{tcpUrg},%{tcpOptions}" } }
            } else if [protocol] == "1" {
                # ICMP
                dissect { mapping => { "restOfLine" => "%{icmpType}},%{restOfLine}" } }
                # Lots more code to dissect each of these ...
            }
        }
    }

```

Note that that code is incomplete and fails to parse UDP correctly because 'if [protocol] in ["6", "17"] {' needs to be two separate branches...

I came up with a different solution... Create a ruby script file that contains

```
def register(params)
    @keys = params['keys']
    @values = params['values']
end

def filter(event)
  values = event.get(@values)
  @keys.each { |k|
    event.set(k, values.shift)
  }
  event.set(@values, values)
[event]
end

```

You call that using a ruby filter like

```
        ruby {
            path => "/home/user/addFields.rb"
            script_params => {
                "values" => "restOfLine"
                "keys" => ["tos", "ecn", "ttl", "id", "offset", "flags", "protocol", "protocolName"]
            }
        }

```

That assumes that restOfLine is an array that contains values. It will remove entries from the array for each of the keys and add them to the event.

Using that like this

```
    dissect { mapping => { "message" => "<%{syslog_pri}>%{[@metadata][ts]} %{+[@metadata][ts]} %{+[@metadata][ts]} filterlog: %{restOfLine}" } }
    date { match => ["[@metadata][ts]", "MMM dd HH:mm:ss" ] }

    mutate { split => { "restOfLine" => "," } }

    ruby {
        path => "/home/user/addFields.rb"
        script_params => {
            "values" => "restOfLine"
            "keys" => ["rule", "subrule", "anchor", "tracker", "interface", "reason", "action", "direction", "ipVersion"]
        }
    }

    if [ipVersion] == "4" {
        ruby {
            path => "/home/user/addFields.rb"
            script_params => {
                "values" => "restOfLine"
                "keys" => ["tos", "ecn", "ttl", "id", "offset", "flags", "protocol", "protocolName"]
            }
        }
    } else if [ipVersion] == "6" {
        # Never seen one of these, "protocol" could be the name, in which case protocol and protocolId should change to be like V4
        ruby {
            path => "/home/user/addFields.rb"
            script_params => {
                "values" => "restOfLine"
                "keys" => ["class", "flowLabel", "hopLimit", "protocol", "protocolId"]
            }
        }
    } else {
        mutate { add_field => { "parseFailure" => "unknown IP version" } }
    }

    if [ipVersion] in ["4", "6"] {
        ruby {
            path => "/home/user/addFields.rb"
            script_params => {
                "values" => "restOfLine"
                "keys" => ["length", "srcIp", "dstIp"]
            }
        }

        if [protocol] == "1" {
            # ICMP
            ruby {
                path => "/home/user/addFields.rb"
                script_params => {
                    "values" => "restOfLine"
                    "keys" => ["icmpType"]
                }
            }
            if [icmpType] == "0" {
                # Echo reply
                # I am not going to write all these!
            } else if [icmpType] == "3" {
                # Destination unreachable
                # Need to parse the codes etc
            }
            # ... and the other types
        } else if [protocol] == "6" {
            # TCP
            ruby {
                path => "/home/user/addFields.rb"
                script_params => {
                    "values" => "restOfLine"
                    "keys" => ["srcPort", "dstPort", "dataLength", "tcpFlags", "tcpSeq", "tcpAck", "tcpWindow", "tcpUrg", "tcpOptions"]
                }
            }
        } else if [protocol] == "17" {
            # UDP
            ruby {
                path => "/home/user/addFields.rb"
                script_params => {
                    "values" => "restOfLine"
                    "keys" => ["srcPort", "dstPort", "dataLength"]
                }
            }
        } else if [protocol] == "112" {
            # CARP
            ruby {
                path => "/home/user/addFields.rb"
                script_params => {
                    "values" => "restOfLine"
                    "keys" => ["type", "carpTtl", "vhid", "carpVersion", "advskew", "advbase"]
                }
            }
        }
    }
}

```

Will get you udp events like

```
{
      "anchor" => "",
   "interface" => "igb3",
         "tos" => "0x0",
        "rule" => "5",
    "protocol" => "17",
"protocolName" => "udp",
     "dstPort" => "5678",
     "tracker" => "1000000103",
          "id" => "0",
     "srcPort" => "5678",
       "srcIp" => "177.85.233.73",
       "dstIp" => "255.255.255.255",
         "ttl" => "64",
      "offset" => "0",
   "direction" => "in",
      "reason" => "match",
       "flags" => "none",
     "subrule" => "",
     "message" => "<134>Sep 29 09:03:57 filterlog: 5,,,1000000103,igb3,match,block,in,4,0x0,,64,0,0,none,17,udp,134,177.85.233.73,255.255.255.255,5678,5678,114",
      "length" => "134",
    "@version" => "1",
         "ecn" => "",
  "@timestamp" => 2020-09-29T13:03:57.000Z,
  "syslog_pri" => "134",
   "ipVersion" => "4",
      "action" => "block",
  "restOfLine" => [],
  "dataLength" => "114"
}

```

and for tcp events

```
{
      "anchor" => "",
   "interface" => "igb1",
      "tcpSeq" => "2184934504",
         "tos" => "0x0",
        "rule" => "5",
    "protocol" => "6",
"protocolName" => "tcp",
     "dstPort" => "443",
     "tracker" => "1000000103",
          "id" => "10381",
     "srcPort" => "51333",
       "srcIp" => "10.0.5.22",
       "dstIp" => "62.67.238.152",
      "tcpAck" => "",
         "ttl" => "128",
   "tcpWindow" => "8192",
  "tcpOptions" => "mss;nop;wscale;nop;nop;sackOK",
      "offset" => "0",
   "direction" => "in",
      "reason" => "match",
       "flags" => "DF",
     "subrule" => "",
     "message" => "<134>Sep 29 09:03:57 filterlog: 5,,,1000000103,igb1,match,block,in,4,0x0,,128,10381,0,DF,6,tcp,52,10.0.5.22,62.67.238.152,51333,443,0,S,2184934504,,8192,,mss;nop;wscale;nop;nop;sackOK",
      "length" => "52",
      "tcpUrg" => "",
    "@version" => "1",
         "ecn" => "",
  "@timestamp" => 2020-09-29T13:03:57.000Z,
  "syslog_pri" => "134",
   "ipVersion" => "4",
      "action" => "block",
  "restOfLine" => [],
  "dataLength" => "0",
    "tcpFlags" => "S"
}

```

I leave it as an exercise for the reader to add IPV6 handling, ICMP handling, and error handling.

---

<div class="post-metadata">

**Author:** ![Lusca\_lusca](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lusca_lusca/32/45091_2.png) [@Lusca\_lusca](https://discuss.elastic.co/u/Lusca_lusca)\
**Post date:** [September 30, 2020, 11:43am UTC](https://discuss.elastic.co/t/configure-logstash-to-show-logs-from-port-5002/250205/19 "2020-09-30T11:43:52Z")

</div>

Thank you so much, I am going to try that now and I will post the results afterwards

---

<div class="post-metadata">

**Author:** ![Lusca\_lusca](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lusca_lusca/32/45091_2.png) [@Lusca\_lusca](https://discuss.elastic.co/u/Lusca_lusca)\
**Post date:** [September 30, 2020, 12:36pm UTC](https://discuss.elastic.co/t/configure-logstash-to-show-logs-from-port-5002/250205/20 "2020-09-30T12:36:02Z")

</div>

This is my configuration now, but I still can't see any IPs. I do want to do the job properly, but firset I would like to see it working by showing the IPs on kibana. Is there something wrong with this?

```
input {
  udp {
    type => "syslog"
    port => 5002
  }
}

filter{
        grok { match => { "restOfLine" => "%{IP:srcIP},%{IP:dstIP}" } }
}
```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [September 30, 2020, 12:57pm UTC](https://discuss.elastic.co/t/configure-logstash-to-show-logs-from-port-5002/250205/21 "2020-09-30T12:57:57Z")

</div>

You have not used dissect to create the restOfLine field. You could use grok to match against the [message] field.

---

<div class="post-metadata">

**Author:** ![Lusca\_lusca](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lusca_lusca/32/45091_2.png) [@Lusca\_lusca](https://discuss.elastic.co/u/Lusca_lusca)\
**Post date:** [October 1, 2020, 11:44am UTC](https://discuss.elastic.co/t/configure-logstash-to-show-logs-from-port-5002/250205/22 "2020-10-01T11:44:56Z")

</div>

still, completly unresponsive, I think there might be something wrong with some configuration. I can see every logs from my server, but when it comes to the remote pfsense firewall, nothing

[Next page](https://discuss.elastic.co/t/configure-logstash-to-show-logs-from-port-5002/250205.md?page=2)