Geoip.location not being created

Elastic Stack Elasticsearch
1

I'm looking to use the geoip.location field but it's not being created. I've found a ton of articles about updating it to a geo_point type but I'm not finding any about the issue of it simply not being created. Odd thing is, every OTHER geoip field is there, just not geoip.location

This is an out of the box, all v. 6.0.1 install: Elasticsearch, Logstash, Filebeat, Metricbeat. Here's what I've done thus far:

  • Installed the ES ingest-geoip plugin (and uninstalled and reinstalled)
  • Loaded, reloaded, overwritten, deleted, and re-loaded the index template
  • Refreshed the index mappings
  • Made sure the apache2 module in filebeat is enabled

But all the other geoip fields are there, including geoip.location.lat and geoip.location.lon. Just not the field that combines the two as a geo_point type.

Further puzzling me, in the JSON data I've got:

"_source": {
"request": "/index.html",
"agent": ""Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36"",
"geoip": {
"timezone": "America/Los_Angeles",
"ip": "xxx.xxx.xxx.242",
"latitude": 32.6361,
"continent_code": "NA",
"city_name": "Chula Vista",
"country_name": "United States",
"country_code2": "US",
"dma_code": 825,
"country_code3": "US",
"region_name": "California",
"location": {
"lon": -117.0532,
"lat": 32.6361
},
"postal_code": "91910",
"region_code": "CA",
"longitude": -117.0532
},

So then I went back into the filebeat.yml and _source.enable: false is commented out under setup.template.settings - it's not turned off (though if _source was turned off it wouldn't be in the JSON data anyway, I would think.) Anyway, so now I'm kinda stuck.

I've also got:
geoip {
source => "clientip"
}

as part of my apache logstash filter.

And I had all this working just fine in a 5.6.4 install on another machine.

Still relatively new to ES and it's components and while I'm fine with things not working - you never learn anything if something doesn't break, I've found - I also know when it's time to ask for help.

Any help would be appreciated!

Thanks in advance!

2

The geoip.location field is there. It is created as an object that contains fields lat and lon. What else did you expect?

3

What does your mappings for the index look like? Does the index template apply to the index the data is being written to?

4

I expected version 6 to work as well as version 5.6 did. Especially when using non-modified, straight out of the box configurations and template files.

geoip.location is in the _source area but is not being passed to the geoip.location field (see below):

geoip
geoip.jpg750×501 45 KB

Screenshot has different data than listed in the original post but is representative of the issue. Actual IPs were removed prior to uploading.

5

It looks like geoip.location.lat and geoip.location.lon are mapped as numbers. This suggests to me that your index template where geoip.location is mapped to geopoint may not apply to this index. Can you check your index template and verify whether that is the case?

6

Template mappings are included as follows. It looks a little mangled, though, but it's the default template with filebeat 6.0.1 and modules system and apache2 enabled. The template index is for the filebeat index which is where the data is currently being written to:

7

What is the full name of the index the data is being written to?

8

For today's index, the full name is filebeat-6.0.1-2017.12.11

9

I rejiggered a few things, tossed the original logstash filter for the apache2 logs and rewrote it. geoip.location still isn't showing up BUT apache2.access.geoip.location is and that works for what I need. However - and I'll post this in a separate thread - the dashboard plots the locations, but throws an error. Digging in the elasticsearch logs says:

org.elasticsearch.index.query.QueryShardException: failed to find geo_point field [apache2.access.geoip.location]

Anyway, thanks for you help!

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.