Hello Community,
We recently implemented ELK stack using docker compose, I am trying to visualize the palo alto global protect logs for login status and ip address.
Below snip to give idea how I am filtering for system logs in my pipeline, and I am able to receive the fields in CSV filter but for global protect log info is inside the "Description:" field. , How can I further filter this to get the login details from the "Description:" field?
Here is the pipeline
input {
syslog {
timezone => "America/Chicago"
port => "1514"
type => "syslog"
}
}
filter {
if ([message] =~ /SYSTEM/) {
csv {
source => "message"
columns => [
"FUTURE_USE", "ReceiveTime", "SerialNumber", "Type", "Threat_ContentType", "FUTURE_USE",
"GeneratedTime", "VirtualSystem", "EventID", "Object", "FUTURE_USE", "FUTURE_USE", "Module",
"Severity", "Description", "SequenceNumber", "ActionFlags", "DeviceGroupHierarchyLevel1",
"DeviceGroupHierarchyLevel1", "DeviceGroupHierarchyLevel2", "DeviceGroupHierarchyLevel3",
"DeviceGroupHierarchyLevel4", "VirtualSystemName", "DeviceName"
]
}
}
mutate {
# Original message has been fully parsed, so remove it.
remove_field => [ "message" ]
}
# Geolocate logs that have SourceIP if that SourceIP is a non-RFC1918 address
if [SourceIP] and [SourceIP] !~ "(^127\.0\.0\.1)|(^10\.)|(^172\.1[6-9]\.)|(^172\.2[0-9]\.)|(^172\.3[0-1]\.)|(^192\.168\.)|(^169\.254\.)" {
geoip {
source => "SourceIP"
target => "SourceIPGeo"
}
# Delete 0,0 in SourceIPGeo.location if equal to 0,0
if ([SourceIPGeo.location] and [SourceIPGeo.location] =~ "0,0") {
mutate {
replace => [ "SourceIPGeo.location", "" ]
}
}
}
# Geolocate logs that have DestinationIP and if that DestinationIP is a non-RFC1918 address
if [DestinationIP] and [DestinationIP] !~ "(^127\.0\.0\.1)|(^10\.)|(^172\.1[6-9]\.)|(^172\.2[0-9]\.)|(^172\.3[0-1]\.)|(^192\.168\.)|(^169\.254\.)" {
geoip {
source => "DestinationIP"
target => "DestinationIPGeo"
}
# Delete 0,0 in DestinationIPGeo.location if equal to 0,0
if ([DestinationIPGeo.location] and [DestinationIPGeo.location] =~ "0,0") {
mutate {
replace => [ "DestinationIPGeo.location", "" ]
}
}
}
if [SourceIP] and [DestinationIP] {
fingerprint {
concatenate_sources => true
method => "SHA1"
key => "logstash"
source => [ "SourceIP", "SourcePort", "DestinationIP", "DestinationPort", "Protocol" ]
}
}
}
output {
elasticsearch {
hosts => "http://URL"
user => *****
password => *******
index => "paloaltologs-%{+YYYY.MM.dd}"
}
stdout { codec => rubydebug }
}
Here is the original log format for system logs of palo alto
"FUTURE_USE", "ReceiveTime", "SerialNumber", "Type", "Threat_ContentType", "FUTURE_USE",
"GeneratedTime", "VirtualSystem", "EventID", "Object", "FUTURE_USE", "FUTURE_USE", "Module",
"Severity", "Description", "SequenceNumber", "ActionFlags", "DeviceGroupHierarchyLevel1",
"DeviceGroupHierarchyLevel1", "DeviceGroupHierarchyLevel2", "DeviceGroupHierarchyLevel3",
"DeviceGroupHierarchyLevel4", "VirtualSystemName", "DeviceName"
Sample output data currently I am receiving
EventID: globalprotectportal-auth-succ
VirtualSystem: -
Description: **GlobalProtect portal user authentication succeeded. Login from: x.x.x.x, Source region: US, User name: ****, Auth type: profile.
VirtualSystemName: PANCorpFW1
type: syslog
GeneratedTime: December 12th 2017, 04:45:57.000
@version: 1
host: 10.3.130.20
SequenceNumber: 1,717,235
Threat_ContentType: globalprotect
timestamp: Dec 12 10:45:57
severity: 6
FUTURE_USE: 0
Severity: informational
priority: 14
logsource:
tags: PA-SYSTEM
Type: SYSTEM
@timestamp: December 12th 2017, 10:45:57.000
DeviceGroupHierarchyLevel4: -
SerialNumber:
DeviceGroupHierarchyLevel3: 0
DeviceGroupHierarchyLevel2: 0
DeviceGroupHierarchyLevel1: 0
ReceiveTime: December 12th 2017, 04:45:57.000
Object: GP-Portal
ActionFlags: 0x0
Module: general
facility: 1
severity_label: Informational
indent preformatted text by 4 spaces
Now I want to visualize the information in the Description field like username, Public IP, region.
Thanks in advance