Geoip.location not Auto Generated

Boris_Joseph · December 8, 2020, 5:28am

hi,

I am shipping & Parsing access logs for Apache webserver using beats and logstash.

Added webUserID as source for GeoIP in logstash and refreshed index pattern.
But geoip.location with geoPoint datatype is not showing up in the index fields(all other geoip fields are).

Not using any index templates. And Logstash creates Index everyday based on date regex.

Can anyone help figure out why geoip.location with geoPoint datatype is not auto-generated?

Should i add a custom field in geoIP via logstash combining geopi.lat +geoip.lon and create index template mapping custom field as geo_point type?

#################
Logstash GeoIP Config
#################
'geoip{

source => "webUserID"*
}'

###############
Index Mapping
###############
'"geoip": {

```
     "properties": {            *
```
```
       "latitude": {*
```
```
         "type": "float"*
```
```
       },*
```
```
       "location": {*
```
```
         "properties": {*
```
```
           "lat": {*
```
```
             "type": "float"*
```
```
           },*
```
```
           "lon": {*
```
```
             "type": "float"*
```
```
           }*
```
```
         }*
```
```
       },*
```
```
       "longitude": {*
```
```
         "type": "float"*
```
```
       }            *
```
```
     }*
```
```
   }*'
```

###############
Kibana Discover
###############

warkolm · December 8, 2020, 6:49am

That is likely why, you need to set it as a a geopoint in a template.

Boris_Joseph · December 8, 2020, 8:05am

Ok. Will try with template and check results.

Just frustrated that ES can't identify/construct geoip.location with geopoint data type., when geoip.location.lat and geoip.location.lon is already present.

Extra step to define/modify geoip fields, just breaks the illusion of single config model for geoip.

Boris_Joseph · December 9, 2020, 4:46am

defined index template as follows and got below error.
Any clue?

warkolm · December 9, 2020, 9:21pm

Kibana doesn't like to mix mapping types. If you want it fixed you will want to reindex the old data, or just delete it if it's not important.

Boris_Joseph · December 10, 2020, 6:45am

Added a custom field in geoip combining latitude and longitude in logsatsh.

Defined index template with custom field defined as geopoint.

And deleted old indexes and refreshed index pattern.
Worked.

Though I still think that when adding geoip to logstash first time, and refreshing existing index pattern, should auto identify geoip.location as geopoint type; since field would be generated for the first time; even though indices may already exist.

system · January 7, 2021, 6:45am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.