Geoip.location.lat and geoip.location.lon showing as number, cannot convert to geoip type

Elastic Stack Logstash
1

I actually want to make a user-location based map in kibana, but I cannot do that because geoip.location.lat and geoip.location.lon showing as number, I cannot convert to geoip type.

this is my logstash conf

filter {
        if "abc" in [tags] {
				grok {
						match => { "message" => '\A%{IPORHOST:clientip} %{HTTPDUSER:remote_logical_user} %{HTTPDUSER:authenticated_user} \[%{HTTPDATE:timestamp}\] "%{WORD:method} %{NOTSPACE:request} (?<http_version>[a-zA-Z\/\d\.]+)" %{NUMBER:response_code} (?:%{NUMBER:bytes_sent}|-)' }
				}

				date {
						match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
				}
				
				geoip {
						source => "clientip"
				}
        }

}

my filter is doing good and I can discover following fields related to geoip

geoip.continent_code
geoip.country_code2
geoip.country_code3
geoip.country_name
geoip.ip
geoip.latitude
geoip.location.lat
geoip.location.lon
geoip.longitude
geoip.timezone

2

Did you rename your output index?

3

no, I am sending the index name by filebeat. No changes made in logstash.

4

Are you sending Filebeat > Logstash? And then sending to Elasticsearch with no changes to the defaults?

5

yes

this is my filebeat conf

- type: log
  enabled: true
  paths:
    - /abc/localhost_access_log.*.txt
  fields_under_root: true
  fields:
    index: ["logstash-abc"]
  tags: ["abc"]
6

can anyone help?

7

What is your index name?

8

@mostafa_kamal - you may need to include the add_field option. See below:

filter {
        if "abc" in [tags] {
				grok {
						match => { "message" => '\A%{IPORHOST:clientip} %{HTTPDUSER:remote_logical_user} %{HTTPDUSER:authenticated_user} \[%{HTTPDATE:timestamp}\] "%{WORD:method} %{NOTSPACE:request} (?<http_version>[a-zA-Z\/\d\.]+)" %{NUMBER:response_code} (?:%{NUMBER:bytes_sent}|-)' }
				}

				date {
						match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
				}
				
				geoip {
						source => "clientip"
                                                add_field => {
                                     "[geoip][enteryourfieldname]" => "%{[geoip][latitude]},%{[geoip][longitude]}"
                                                }
			         }
        }

}

The add_field option will create a new field and combine the latitude and longitude values to the new field.

9

@Feedy, I have added add_field option. But that also shows like a string.

				geoip {
						source => "clientip"
                                add_field => {
										"[geoip][geoip_custom_location]" => "%{[geoip][latitude]},%{[geoip][longitude]}"
                                             }
			         }

How to create map with that?

10

@mostafa_kamal
Please show your field mapping for this index. You may need to update this index mapping to reflect the new field. Here is a screenshot of the field mapping for my index(where you see "coordinates" is the same as your "geoip_custom_location" I just choose to use a different name):

index_mapping_geoip
index_mapping_geoip785×610 30.8 KB

11

@Feedy, thanks for your feedback.
Actually, I did kinda the same thing.

I have created a template and convert the field to geo_point. I have created a new index and it's solved now.

PUT _template/abc
{
"index_patterns": [
  "abc"
  ],
"settings": { },
 "mappings": {
   "properties": {
     "def" : {
       "type": "geo_point"
        }
      }
    }, 
"aliases": { }
}

Thank you.

12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.