Map geoip.location to geo_point by default

Elastic Stack Logstash
1

I'm trying to visualise geoip location on a tile map in Kibana but am getting the following error:

No Compatible Fields: The "[logstash-nginx-access-]YYYY.MM.DD" index pattern does not contain any of the following field types: geo_point

I found this issue on GitHub that describes the exact problem. It proposes a solution of prefixing the index with logstash-, which is what the template matches on; however, my index is already prefixed correctly:

output {
    if [type] == "nginx_access" {
        elasticsearch {
            hosts => "localhost:9200"
            sniffing => true
            manage_template => false
            index => "logstash-nginx-access-%{+YYYY.MM.dd}"
            document_type => "%{[@metadata][type]}"
        }
    }
}

Is anything else I need to get logstash to use the default template for elasticsearch or another way to have geoip data correctly mapped?

2

Have you reloaded the fields in Kibana? Have you checked how location.geoip actually is mapped for the index in question (use the get mapping API)?

3

Yes, I've tried reloading the index in Kibana. I've also removed the setting and re-added it, but in both cases the geoip fields weren't mapped to geo_point.

Running curl 'localhost:9200/logstash-nginx-access-2015.11.19/_mapping?pretty=true outputs the following. You can see that the geoip.location field is given a double type, as in the GitHub issue.

{
  "logstash-nginx-access-2015.11.19" : {
    "mappings" : {
      "nginx_access" : {
        "properties" : {
          "@timestamp" : {
            "type" : "date",
            "format" : "strict_date_optional_time||epoch_millis"
          },
          "@version" : {
            "type" : "string"
          },
          "agent" : {
            "type" : "string"
          },
          "auth" : {
            "type" : "string"
          },
          "bytes" : {
            "type" : "string"
          },
          "clientip" : {
            "type" : "string"
          },
          "count" : {
            "type" : "long"
          },
          "fileinfo" : {
            "type" : "object"
          },
          "geoip" : {
            "properties" : {
              "area_code" : {
                "type" : "long"
              },
              "city_name" : {
                "type" : "string"
              },
              "continent_code" : {
                "type" : "string"
              },
              "country_code2" : {
                "type" : "string"
              },
              "country_code3" : {
                "type" : "string"
              },
              "country_name" : {
                "type" : "string"
              },
              "dma_code" : {
                "type" : "long"
              },
              "ip" : {
                "type" : "string"
              },
              "latitude" : {
                "type" : "double"
              },
              "location" : {
                "type" : "double"
              },
              "longitude" : {
                "type" : "double"
              },
              "real_region_name" : {
                "type" : "string"
              },
              "region_name" : {
                "type" : "string"
              },
              "timezone" : {
                "type" : "string"
              }
            }
          },
          "httpversion" : {
            "type" : "string"
          },
          "ident" : {
            "type" : "string"
          },
          "input_type" : {
            "type" : "string"
          },
          "line" : {
            "type" : "long"
          },
          "message" : {
            "type" : "string"
          },
          "offset" : {
            "type" : "long"
          },
          "referrer" : {
            "type" : "string"
          },
          "request" : {
            "type" : "string"
          },
          "response" : {
            "type" : "string"
          },
          "shipper" : {
            "type" : "string"
          },
          "source" : {
            "type" : "string"
          },
          "tags" : {
            "type" : "string"
          },
          "timestamp" : {
            "type" : "string"
          },
          "type" : {
            "type" : "string"
          },
          "verb" : {
            "type" : "string"
          }
        }
      }
    }
  }
}
4

You've set manage_template to false, so are you in fact managing the templates yourself? Is there a template in ES that will be applied to logstash-nginx-access-2015.11.19? Check with the index template APIs. If things look okay there, what happens if you create, say, logstash-nginx-access-2015.12.01? Will its mappings look alright?

5

manage_template was the problem, thanks. It's not clear at first glance who is managing the template when that's set to true. I assumed setting it to true meant that I was managing it, which I didn't want. Thanks.