GeoIP dosent work on elk stack( filebeat->logstash->ES->kibana) [FIXED]

Elastic Stack Elasticsearch
1

Hi,
I'm used this tutorial to setup my elk stack - How To Install Elasticsearch, Logstash, and Kibana (ELK Stack) on CentOS 7 | DigitalOcean.
I cant seem to get geoip to work,. when I creater a tile map visualiztion I get:

No Compatible Fields: The "filebeat-*" index pattern does not contain any of the following field types: geo_point

This is my log stash configuration:

input {
  beats {
    port => 5044
    ssl => true
    ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
    ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
  }
}
    **11-nginx-filter.conf** 
    filter {
      if [type] == "nginx-access" {
        grok {
          match => { "message" => "%{NGINXACCESS}" }
        }
      }
     geoip {
          source => "clientip"
          target => "geoip"
          database => "/etc/logstash/GeoLiteCity.dat"
          add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ]
          add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}"  ]
        }
        mutate {
          convert => [ "[geoip][coordinates]", "float"]
        }
   ...
    }
      elasticsearch {
        hosts => ["localhost:9201"]
        sniffing => true
        manage_template => false
        index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
        document_type => "%{[@metadata][type]}"
        template => "/etc/logstash/templates/elasticsearch-template-filebeat.json"
        template_name => "filebeat"
      }
    }

this is my template file:
/etc/logstash/templates/elasticsearch-template-filebeat.json

{
    "template": "filebeat-*",
    "settings": {
    "index.refresh_interval": "5s"
},
    "mappings": {
    "_default_": {
        "_all": {
            "enabled": true,
                "omit_norms": true
        },
        "dynamic_templates": [{
            "message_field": {
                "match": "message",
                "match_mapping_type": "string",
                "mapping": {
                    "type": "string",
                    "index": "analyzed",
                    "omit_norms": true,
                    "fielddata": {
                        "format": "disabled"
                    }
                }
            }
        }, {
       
            "geoip": {
                "dynamic": true,
                    "properties": {
                    "ip": {
                        "doc_values": true,
                        "type": "ip"
                    },
                    "location": {
                        "doc_values": true,
                        "type": "geo_point"
                    },
                    "latitude": {
                        "doc_values": true,
                        "type": "float"
                    },
                    "longitude": {
                        "doc_values": true,
                        "type": "float"
                    }
                }
            }
        }
    }
}
}

and mapping:
/filebeat-2016.09.05/_mapping?pretty

{
  "filebeat-2016.09.05" : {
  ...
          "geoip" : {
            "dynamic" : "true",
            "properties" : {
              "area_code" : {
                "type" : "long"
              },
              ...
              "coordinates" : {
                "type" : "double"
              },
             ...
              "location" : {
                "type" : "geo_point"
              },

Does anyone know I can I fix it?
Thanks,
Dan

1 Like
2

I've managed to fix this.
all i had to do is refresh filebeat indexing in kibana ( setting-> refresh)

2 Likes