Hi all,
I am running Elastic,Kibana and Filebeat 8.8.1 on docker and it somewhat works for what i want to achieve, but i am struggling to get Juniper module to ingest my data correctly.
From what i see at : https://github.com/elastic/beats/blob/master/x-pack/filebeat/module/juniper/srx/test/flow.log it looks quiet different from the data im getting in from my SRX.
What am i missing? How can i correct it?
Example:
<14>1 2019-11-14T09:37:51.184+01:00 SRX-GW1 RT_FLOW - RT_FLOW_SESSION_CREATE [junos@2636.81.2.69.144.134 source-address="10.0.0.1" source-port="594" destination-address="10.128.0.1" destination-port="10400" connection-tag="0" service-name="icmp" nat-source-address="10.0.0.1" nat-source-port="594" nat-destination-address="10.128.0.1" nat-destination-port="10400" nat-connection-tag="0" src-nat-rule-type="N/A" src-nat-rule-name="N/A" dst-nat-rule-type="N/A" dst-nat-rule-name="N/A" protocol-id="1" policy-name="vpn_trust_permit-all" source-zone-name="vpn" destination-zone-name="trust" session-id-32="6093" username="N/A" roles="N/A" packet-incoming-interface="st0.0" application="UNKNOWN" nested-application="UNKNOWN" encrypted="UNKNOWN" application-category="N/A" application-sub-category="N/A" application-risk="1" application-characteristics="N/A"]
<14>1 2019-11-14T11:12:46.573+01:00 SRX-GW1 RT_FLOW - RT_FLOW_SESSION_DENY [junos@2636.81.2.69.144.134 source-address="10.0.0.26" source-port="37233" destination-address="10.128.0.1" destination-port="161" connection-tag="0" service-name="None" protocol-id="17" icmp-type="0" policy-name="MgmtAccess-trust-cleanup" source-zone-name="trust" destination-zone-name="junos-host" application="UNKNOWN" nested-application="UNKNOWN" username="N/A" roles="N/A" packet-incoming-interface=".local..0" encrypted="No" reason="Denied by policy" session-id-32="7087" application-category="N/A" application-sub-category="N/A" application-risk="1" application-characteristics="N/A"]
<14>1 2014-05-01T08:26:51.179Z fw01 RT_FLOW - RT_FLOW_SESSION_DENY [junos@2636.81.2.69.144.39 source-address="81.2.69.143" source-port="56639" destination-address="5.6.7.8" destination-port="2003" service-name="None" protocol-id="6" icmp-type="0" policy-name="log-all-else" source-zone-name="campus" destination-zone-name="mngmt" application="UNKNOWN" nested-application="UNKNOWN" username="N/A" roles="N/A" packet-incoming-interface="reth6.0" encrypted="No "]
What im typically receiving:
<14>Aug 1 15:38:40 SFGSFW01 RT_FLOW: RT_FLOW_SESSION_CREATE: session created 172.16.10.27/46192->10.101.2.12/12000 0x0 None 172.16.10.27/46192->10.101.2.12/12000 0x0 N/A N/A N/A N/A 6 some-policy-name_201 another-zone some-zone 115964325984 N/A(N/A) ge-0/0/2.205 UNKNOWN UNKNOWN UNKNOWN N/A N/A -1 N/A N/A N/A Off root N/A N/A
<14>Aug 1 15:38:40 SFGSFW01 RT_FLOW: RT_FLOW_SESSION_CLOSE: session closed TCP FIN: 172.16.10.27/25845->10.101.2.11/80 0x0 junos-http 172.16.10.27/25845->10.101.2.11/80 0x0 N/A N/A N/A N/A 6 some-policy-name_201 another-zone some-zone 115964321050 5(277) 7(2284) 2 UNKNOWN UNKNOWN N/A(N/A) ge-0/0/2.205 UNKNOWN N/A N/A -1 N/A NA 0 0.0.0.0/0->0.0.0.0/0 NA NA N/A N/A Off root 0 N/A N/A N/A
filebeat.yml:
filebeat.config:
modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: true
filebeat.modules:
- module: juniper
srx:
enabled: true
var.input: udp
var.syslog_host: 0.0.0.0
var.syslog_port: 5140
logging:
level: debug
output.elasticsearch:
hosts: '${ELASTICSEARCH_HOSTS:192.168.10.70:9200}'
setup.ilm.overwrite: true
setup.dashboards:
enabled: true
setup.kibana:
host: "http://localhost:5601"
Another thing im wondering about is when you are running filebeat on docker are you supposed to see the enabled modules from within the filebeat docker?
root@docker:/usr/share/filebeat# docker exec filebeat filebeat modules list
Enabled:
Disabled:
activemq
apache
auditd
aws
awsfargate
azure
barracuda
bluecoat
cef
checkpoint
cisco
coredns
crowdstrike
cyberarkpas
cylance
elasticsearch
envoyproxy
f5
fortinet
gcp
google_workspace
haproxy
ibmmq
icinga
iis
imperva
infoblox
iptables
juniper
.....