# Logstash + Fortinet + Kibana
**URL:** https://discuss.elastic.co/t/logstash-fortinet-kibana/323508
**Category:** Kibana
**Created:** [January 19, 2023, 12:15pm UTC](https://discuss.elastic.co/t/logstash-fortinet-kibana/323508 "2023-01-19T12:15:05Z")
**Posts on this page:** 12
**Page:** 1
### Author:  [@Cezary](https://discuss.elastic.co/u/Cezary)
#### Post date: [January 19, 2023, 12:15pm UTC](https://discuss.elastic.co/t/logstash-fortinet-kibana/323508/1 "2023-01-19T12:15:05Z")
Hello to All,
I'm trying to create Kibana map using data from Fortinet syslog and Logstash.
I was able to load geoip data to kibana, however geo.location field had to be created from Logstash because it was not created automatically.
My Logstash config looks like this at the moment:
```auto
input {
udp {
port => 5004
type => fortinet
}
}
filter {
if [type] == "fortinet" {
grok {
match => {"message" => "%{SYSLOG5424PRI}%{GREEDYDATA:message}" }
overwrite => ["message"]
}
mutate {
remove_field => ["@timestamp","@version","event","log"]
}
kv {
source => "message"
value_split => "="
field_split => ","
remove_field => "message"
}
mutate {
rename => { "type" => "ftg_type" }
rename => { "subtype" => "ftg_subtype" }
add_field => { "type" => "fortinet" }
add_field => { "logdate" => "%{date} %{time}" }
convert => { "rcvdbyte" => "integer" }
convert => { "sentbyte" => "integer" }
}
date {
match => ["logdate", "yyyy-MM-dd HH:mm:ss"]
timezone => "Europe/Paris"
target => "@timestamp"
}
mutate {
remove_field => ["date","time"]
}
date_formatter {
source => "@timestamp"
target => "log_day"
pattern => "YYYY.MM.dd"
}
if [srcip] {
if [srcip] !~ /^(10.|[a-f])/ {
geoip {
source => "srcip"
target => "src_geoip"
}
if [src_geoip] {
mutate {
add_field => ["[src_geoip][location]", "%{[src_geoip][geo][location][lat]}, %{[src_geoip][geo][location][lon]}" ]
}
}
}
}
if [dstip] {
if [dstip] !~ /^(10.|[a-f])/ {
geoip {
source => "dstip"
target => "dst_geoip"
}
if [dst_geoip] {
mutate {
add_field => ["[dst_geoip][location]", "%{[dst_geoip][geo][location][lat]}, %{[dst_geoip][geo][location][lon]}" ]
}
}
}
}
}
}
output {
if [type] == "fortinet" {
elasticsearch {
hosts => ["https://localhost:9200"]
user => "logstash_internal"
password => ""
ssl => true
ssl_certificate_verification => false
index => "fortinet-%{+YYYY.MM.dd}"
}
file {
path => "/log/%{log_day}/fortinet/%{devname}/%{devname}-%{[host][ip]}.gzip"
gzip => true
}
}
}
```
Also modified template for this index in Kibana:
```auto
{
"properties": {
"dst_geoip.location": {
"ignore_malformed": false,
"type": "geo_point",
"ignore_z_value": false
},
"src_geoip.location": {
"ignore_malformed": false,
"type": "geo_point",
"ignore_z_value": false
}
}
}
```
Anyway, I can see this fields in Kibana:
But when I see mappings in specific index it is mapped twice:
And field is still displayed as text and can't be selected as geopoint in the MAP:
Can anyone help me with that issue?
Best regards.
---
### Author:  [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [January 19, 2023, 1:06pm UTC](https://discuss.elastic.co/t/logstash-fortinet-kibana/323508/2 "2023-01-19T13:06:15Z")
A field can not be mapped twice, so there is something wrong in your configuration, can you share the entire `src_geoip` and `dst_geoip` mapping as plain text using the preformatted text option, not as image?
Probably this is the issue:
```auto
{
"properties": {
"dst_geoip.location": {
"ignore_malformed": false,
"type": "geo_point",
"ignore_z_value": false
},
"src_geoip.location": {
"ignore_malformed": false,
"type": "geo_point",
"ignore_z_value": false
}
}
}
```
This creates a mapping for a field `dst_geop.location` where the dot is parte of the name, this creates a mapping for the following field:
```auto
{ "dst_geoip.location": "value" }
```
What you want is:
```auto
{ "dst_geoip": { "location": "value" } }
```
---
### Author:  [@Rios](https://discuss.elastic.co/u/Rios)
#### Post date: [January 19, 2023, 10:43pm UTC](https://discuss.elastic.co/t/logstash-fortinet-kibana/323508/3 "2023-01-19T22:43:44Z")
Since LS v8+ is ECSv8 is by default, the geoip plugin will work if you set the field name in format: [source][ip], like this:
```auto
geoip {
source => "[source][ip]"
ecs_compatibility => "v8" # default
tag_on_failure => ["Location unknown or similar msg"]
}
```
Similar is for [destination][ip] for [the destination](https://www.elastic.co/guide/en/ecs/current/ecs-destination.html#field-destination-ip)
If you set _[ecs\_compatibility](https://www.elastic.co/guide/en/logstash/current/plugins-filters-geoip.html#plugins-filters-geoip-ecs_compatibility) = disabled_ then you can lookup directly the
srcip or dstip fields.
1. For custom geo data you have to create GeoJSON, [ECS v8](https://www.elastic.co/guide/en/ecs/current/ecs-geo.html) which should look like this.
```auto
if ([srcip] =~ /^10\./) {
mutate {
add_field => {
"[src_geoip][city_name]" => "Baltimore"
"[src_geoip][continent_code]" => "NA"
"[src_geoip][continent_name]" => "North America"
"[src_geoip][country_iso_code]" => "US"
"[src_geoip][country_name]" => "USA"
"[src_geoip][location][lon]" => -76.6348
"[src_geoip][location][lat]" => 39.2851
"[src_geoip][name]" => "Baltimore HQ"
"[src_geoip][postal_code]" => "667"
"[src_geoip][region_iso_code]" => "US-MD"
"[src_geoip][region_name]" => "Maryland"
"[src_geoip][timezone]" => "America/Detroit"
}
}
}
```
ECS v1 looks very similar, check what your LS which the structure will generate.
1. You must define mappings-dataview. FB is using dynamic structure
```auto
"src_geoip": {
"properties": {
"region_iso_code": {
"ignore_above": 1024,
"type": "keyword"
},
"continent_name": {
"ignore_above": 1024,
"type": "keyword"
},
"city_name": {
"ignore_above": 1024,
"type": "keyword"
},
"country_iso_code": {
"ignore_above": 1024,
"type": "keyword"
},
"timezone": {
"ignore_above": 1024,
"type": "keyword"
},
"country_name": {
"ignore_above": 1024,
"type": "keyword"
},
"name": {
"ignore_above": 1024,
"type": "keyword"
},
"continent_code": {
"ignore_above": 1024,
"type": "keyword"
},
"region_name": {
"ignore_above": 1024,
"type": "keyword"
},
"location": {
"type": "geo_point"
},
"postal_code": {
"ignore_above": 1024,
"type": "keyword"
}
}
},
```
For older versions,which include country\_code2, country\_code3 etc., you can use [the structure:](https://www.elastic.co/guide/en/logstash/current/plugins-filters-geoip.html#plugins-filters-geoip-field-mapping)
```auto
"src_geoip" : {
"dynamic": true,
"properties" : {
"ip": { "type": "ip" },
"location" : { "type" : "geo_point" },
"latitude" : { "type" : "half_float" },
"longitude" : { "type" : "half_float" }
}
},
```
---
### Author:  [@Cezary](https://discuss.elastic.co/u/Cezary)
#### Post date: [February 6, 2023, 1:09pm UTC](https://discuss.elastic.co/t/logstash-fortinet-kibana/323508/4 "2023-02-06T13:09:17Z")
Hi,
Sorry for the delay, I had a winter break and some more time took to verify all suggestions.
At he mement my geoip config looks like this:
```auto
mutate {
rename => { "srcip" => "[source][ip]" }
rename => { "dstip" => "[destination][ip]" }
}
if [source][ip] !~ /^(10.|[a-f]|255.255.255.255)/ {
geoip {
source => "[source][ip]"
ecs_compatibility => "v8"
tag_on_failure => ["Location unknown"]
}
}
if [destination][ip] !~ /^(10.|[a-f]|255.255.255.255)/ {
geoip {
source => "[destination][ip]"
ecs_compatibility => "v8"
tag_on_failure => ["Location unknown"]
}
```
But in descovery I can see many geo data but not the geo\_point location:
Also no geo+point available when trying to create map. I tried to create template for index template used by that index but it basically it resaults in missing geo data.
At the moment mappings looks like this (defaults):
```auto
"source": {
"properties": {
"geo": {
"properties": {
"city_name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"continent_code": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"country_iso_code": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"country_name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"location": {
"properties": {
"lat": {
"type": "float"
},
"lon": {
"type": "float"
}
}
},
"postal_code": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"region_iso_code": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"region_name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"timezone": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
```
Do you have any advice for me what am I doing wrong?
---
### Author:  [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [February 6, 2023, 1:15pm UTC](https://discuss.elastic.co/t/logstash-fortinet-kibana/323508/5 "2023-02-06T13:15:15Z")
Check the previous answers, you need to create the correct mapping before adding any data, you do not have the correct mapping.
---
### Author:  [@Cezary](https://discuss.elastic.co/u/Cezary)
#### Post date: [February 8, 2023, 9:45am UTC](https://discuss.elastic.co/t/logstash-fortinet-kibana/323508/6 "2023-02-08T09:45:34Z")
Hello Guys,
Thank you for your help, it is working now.
Bellow is my config, maybe it will help someone.
Please remember to install date\_formater plugin or remove that section:
```auto
/usr/share/logstash/bin/logstash-plugin install logstash-filter-date_formatter
```
My current config:
```auto
input {
udp {
port => 5004
type => fortinet
}
}
filter {
if [type] == "fortinet" {
grok {
match => {"message" => "%{SYSLOG5424PRI}%{GREEDYDATA:message}" }
overwrite => ["message"]
}
mutate {
remove_field => ["@timestamp","@version","event","log"]
}
kv {
source => "message"
value_split => "="
field_split => ","
remove_field => "message"
}
mutate {
rename => { "type" => "ftg_type" }
rename => { "subtype" => "ftg_subtype" }
rename => { "srcip" => "[source][ip]" }
rename => { "dstip" => "[destination][ip]" }
add_field => { "type" => "fortinet" }
add_field => { "logdate" => "%{date} %{time}" }
convert => { "rcvdbyte" => "integer" }
convert => { "sentbyte" => "integer" }
}
date {
match => ["logdate", "yyyy-MM-dd HH:mm:ss"]
timezone => "Europe/Paris"
target => "@timestamp"
}
mutate {
remove_field => ["date","time"]
}
date_formatter {
source => "@timestamp"
target => "log_day"
pattern => "YYYY.MM.dd"
}
if [source][ip] !~ /^(10.|[a-f]|255.255.255.255)/ {
geoip {
source => "[source][ip]"
ecs_compatibility => "v8"
tag_on_failure => ["Location unknown"]
}
}
if [destination][ip] !~ /^(10.|[a-f]|255.255.255.255)/ {
geoip {
source => "[destination][ip]"
ecs_compatibility => "v8"
tag_on_failure => ["Location unknown"]
}
}
}
}
output {
if [type] == "fortinet" {
elasticsearch {
hosts => ["https://localhost:9200"]
user => "logstash_internal"
password => "some password"
ssl => true
ssl_certificate_verification => false
index => "fortinet-%{+YYYY.MM.dd}"
}
file {
path => "/log/%{log_day}/fortinet/%{devname}/%{devname}-%{[host][ip]}.gzip"
gzip => true
}
}
}
```
You need also to add index template:
```auto
"properties": {
"source.geo.location": {
"type": "geo_point"
},
"destination.geo.location": {
"type": "geo_point"
}
```
In kibana gui it should look like this:
---
### Author:  [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [February 8, 2023, 12:18pm UTC](https://discuss.elastic.co/t/logstash-fortinet-kibana/323508/7 "2023-02-08T12:18:30Z")
> [@Cezary](#):
>
> ```auto
> "properties": {
> "source.geo.location": {
> "type": "geo_point"
> },
> "destination.geo.location": {
> "type": "geo_point"
> }
>
> ```
This mapping is wrong, this creates mappings for fields with literal dots in the name, not for json objects.
---
### Author:  [@Cezary](https://discuss.elastic.co/u/Cezary)
#### Post date: [February 8, 2023, 12:49pm UTC](https://discuss.elastic.co/t/logstash-fortinet-kibana/323508/8 "2023-02-08T12:49:41Z")
@leandrojmp
Information that it is wrong does not solve the issue- what is the correct one?
Here it is how it look like from my point of view:
1. Template text mapping view:
2. Template gui mapping view:
3. What I see in index created from that template:
```auto
"destination": {
"properties": {
"geo": {
"properties": {
"city_name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"continent_code": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"country_iso_code": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"country_name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"location": {
"type": "geo_point"
},
"postal_code": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"region_iso_code": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"region_name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"timezone": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"source": {
"properties": {
"geo": {
"properties": {
"city_name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"continent_code": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"country_iso_code": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"country_name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"location": {
"type": "geo_point"
},
"postal_code": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"region_iso_code": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"region_name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"timezone": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
```
1. And I can see location points in discovery:
---
### Author:  [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [February 8, 2023, 3:26pm UTC](https://discuss.elastic.co/t/logstash-fortinet-kibana/323508/9 "2023-02-08T15:26:36Z")
> [@Cezary](#):
>
> Information that it is wrong does not solve the issue- what is the correct one?
The correct structure is also on [previous answers](https://discuss.elastic.co/t/logstash-fortinet-kibana/323508/3).
But from what you shared you already has the correct mapping because you have this in your mapping:
```auto
"source": {
"properties": {
"geo": {
"properties": {
"location": {
"type": "geo_point"
}
```
So, you have the correct geo\_point mapping being applied, but it is not coming from the template you shared.
As already said, a mapping like this:
```auto
"source.geo.location": {
"type": "geo_point"
}
```
Will work for a field like this:
```auto
{ "source.geo.location": "value" }
```
This filter in Logstash:
```auto
geoip {
source => "[source][ip]"
ecs_compatibility => "v8"
tag_on_failure => ["Location unknown"]
}
```
Will create the geo location field in the following structure:
```auto
{
"source": {
"geo": {
"location": "value"
}
}
}
```
And the correct mapping for this structure is like the one from the previous linked answer:
```auto
{
"source": {
"properties": {
"geo": {
"properties": {
"location": {"type": "geo_point" }
}
}
}
}
}
```
You didn't share if you are using other component templates or not, but since you have the correct mapping for the location field, this mapping must be coming from another template.
One thing that may lead to confusion is that in Kibana the two following fields will appear as the same:
```auto
{ "source.geo.location": "value" }
```
and
```auto
{
"source": {
"geo": {
"location": "value"
}
}
}
```
Will look as the same in the discover table view, you will only know if you have a field with a dot in the name, as the first one, or a json object, as the second, if you go in the json view of the document.
---
### Author:  [@Cezary](https://discuss.elastic.co/u/Cezary)
#### Post date: [February 9, 2023, 8:56am UTC](https://discuss.elastic.co/t/logstash-fortinet-kibana/323508/10 "2023-02-09T08:56:11Z")
It looks like it is defined some other way in template in kibana - but for sure geo\_point was not visible in data before adding mappings to template in kibana gui.
1. When defining ( **this is the information I'm missing - what should be written there?** ):
It looks wrong in the template but index created from that template looks correctly.
If you want to diagnose it I can make some debuging but I have very litlle expirience with ELK, so things obvious for you might be totaly incomprehensible for me.
---
### Author:  [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [February 10, 2023, 12:57pm UTC](https://discuss.elastic.co/t/logstash-fortinet-kibana/323508/11 "2023-02-10T12:57:21Z")
What do you have in the component templates part?
You may have your mapping comming from a component template.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 10, 2023, 12:57pm UTC](https://discuss.elastic.co/t/logstash-fortinet-kibana/323508/12 "2023-03-10T12:57:49Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.