No location field Geoip plugin

Elastic Stack Logstash
1

After using Geoip plugin in a filter specifying the ip address field I am getting Location.lat and Location.lon fields but I am not getting a location field defined as geopoint, or anything for that matter

image.png976×153 10.4 KB

I have installed the template for my index which is syslog-*

curl -XPUT '10.10.6.60:9200/_template/template_1?pretty' -H 'Content-Type: application/json' -d'
{
"template" : "syslog-",
"version" : 50001,
"settings" : {
"index.refresh_interval" : "5s"
},
"mappings" : {
"default" : {
"_all" : {"enabled" : true, "norms" : false},
"dynamic_templates" : [ {
"message_field" : {
"path_match" : "message",
"match_mapping_type" : "string",
"mapping" : {
"type" : "text",
"norms" : false
}
}
}, {
"string_fields" : {
"match" : "",
"match_mapping_type" : "string",
"mapping" : {
"type" : "text", "norms" : false,
"fields" : {
"keyword" : { "type": "keyword", "ignore_above": 256 }
}
}
}
} ],
"properties" : {
"@timestamp": { "type": "date", "include_in_all": false },
"@version": { "type": "keyword", "include_in_all": false },
"geoip" : {
"dynamic": true,
"properties" : {
"ip": { "type": "ip" },
"location" : { "type" : "geo_point" },
"latitude" : { "type" : "half_float" },
"longitude" : { "type" : "half_float" }
}
},
"location": {
"type": "geo_point"
}
}
}
}
}
'

and here is my conf

input{
beats{
port => 5044
}
tcp{
port => 5151
type => "zywall310"
codec => cef
}
udp{
port => 5151
type => "zywall310"
codec => cef
}
}
filter{
geoip {
source => "sourceAddress"
}
}
output{
if [type] == "zywall310"
{
elasticsearch {
hosts => ["10.10.6.60:9200"]
index => "syslog-%{+YYYY.MM.dd}"
}
}
else
{
elasticsearch {
hosts => ["10.10.6.60:9200"]
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}
}
------------beats.conf------------------

filter{
if [type] == "Zywall310" {
grok {
patterns_dir => ["./patterns"]
match => {
"message" => "%{CISCOTIMESTAMP:timestamp}( %{SYSLOGHOST:sysloghost})? %{WORD:Program}: %{Zywallpri:Priority}|%{Make:make}|%{Model:model}|%{Version:version}|0|%{EventType}|5|src=%{IP:src_ip} dst=%{IP:dst_ip} spt=%{INT:src_port} dpt=%{INT:dst_port} msg=priority:%{Priority}, from %{Source} to %{Destination}, %{WORD:protocol}, service %{WORD:service}, %{Action}"
}
}
}
}
^----------filter.conf----------------^

and my patterns file
Zywallpri [0-9]
Make \w+\b
Model \w+\b\s\d{3}
Version \d+(.\d{2}(\w+.\d))
EventType \w+\s\w+
Priority \d{1,3}
Source \w+
Destination \w+
Action \w+

---------./patterns/Zywall.txt-----------

I'm fairly new to this and am setting it up in my homelab environment just trying to get a working model I can go with so patience is appreciated :slight_smile: . Any help would be great, thx!

2

Where are you defining the sourceAddressc field?

Also please use code formatting, the </> button, to make your posts easier to read :slight_smile:

3

There are 2 filters, one is in Beats.conf which is

filter{
geoip {
source => "sourceAddress"
}
4

Unless I'm missunderstanding you? These are firewall logs, I've gotten everything, including applying the template and recreating the index, but geoip.location never gets created. I have geoip.location.lon and geoip.location.lat just not geoip. I've also ingested ASA logs and get similar results. Any help is appreciated again.

5

I can see you are referring to that field, but unless I am missing something you aren't even creating that field (ie defining it) in your grok patterns, so it'll never be read.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.