Missing Geo_point field in geohash aggregate -! Kibana 4.5

[Bob_Metelsky]

Guys, having a heck of a time creating a simple tile map

all files and configuration are here
https://sites.google.com/site/developtroubleshooting/elastic-geoip

Im having a presentation tomorrow and I need to get a tile map by location.

Under allot of pressure to make this work

Thanks
Bob

[anhlqn (Anh)]

Try replacing your geoip filter in Logstash config

 geoip {
      source => "clientip"
      target => "geoip"
      database => "/etc/logstash/conf.d/GeoLiteCity.dat"
      add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ]
      add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}"  ]
    }
    mutate {
      convert => [ "[geoip][coordinates]", "float"]
    }

with this config

geoip {
            source => "clientip"
            target => "geoip"
}

Your mapping is ok so far, but you may have to delete existing indexes if you see field conflict error.

Start to send logs to Logstash again, and if you see something like this in the stdout, the geoip.location can be used in your Tile map.

geoip.location: [-78.1704, 38.71629999999999]

Make sure that geoip.location is mapped as geo_point in your index.

[anhlqn (Anh)]

Actually, your config seems to work too, as I see the geoip.location field contains appropriate data

"location" => [
            [0] -96.7885,
            [1] 39.08879999999999
        ]

However, the mapping for that field in Elasticsearch according to your link is number. It should be geo_point.

[Bob_Metelsky]

Hi Appreciate the help...

Made the suggested changes to the logstash.conf file, getting the same error

I get the geoip.location

better data same error...

see error image, new conf file and output file
https://sites.google.com/site/developtroubleshooting/elastic-geoip/mapping2

[Bob_Metelsky]

@anhlqn

can be used in your Tile map.

Hi - Also Im not sure what you mean with this...

"Make sure that geoip.location is mapped as geo_point in your index"

How is that done?

[Bob_Metelsky]

Also not sure what you mean with this...
"Make sure that geoip.location is mapped as geo_point in your index"
How is that done?

[anhlqn (Anh)]

@Bob_Metelsky

see error image, new conf file and output filehttps://sites.google.com/site/developtroubleshooting/elastic-geoip/mapping2

I think the issue is that the mappings you specify in your elasticsearch-output-plugin, which is template => "/etc/logstash/conf.d/elasticsearch-apache-weblogs-geoip.json" is not applied to your index named bob

Please try again with this output config for Logstash

output {
stdout { codec => rubydebug }
  elasticsearch {
    index => "apache-weblogs-geoip2"
    hosts => ["t400:9200"]
    manage_template => true
    template => "/etc/logstash/conf.d/elasticsearch-apache-weblogs-geoip.json"
    template_name => "apache-weblogs-geoip"
  }
}

@Bob_Metelsky

Also not sure what you mean with this..."Make sure that geoip.location is mapped as geo_point in your index"How is that done?

I believe your are on Elasticsearch 2.x. If so, you can install the Kopf plugin GitHub - lmenezes/elasticsearch-kopf: web admin interface for elasticsearch to your elasticsearch to add and manage index mapping templates. More about mappings Mapping | Elasticsearch: The Definitive Guide [2.x] | Elastic.

Your current mapping for geo.location is number

but it is supposed to be geo_point like this

[anhlqn (Anh)]

I wrote this guide about geoip a while ago http://www.secureict.info/2015/11/process-netflow-with-nprobe-and_20.html. I hope it clarifies about mapping geo_point fields.