"nginx-access-*" index pattern does not contain any of the following field types: geo_point

Elastic Stack Elasticsearch
1

I get the above error while trying to create tile map visualization based on nginx client ip. I have set up ELK to log my nginx access logs. Have defined a custom index pattern in my filebeat.yml like below

- input_type: log
# Paths that should be crawled and fetched. Glob based paths.
  paths:
    - /var/log/nginx/access.log
    document_type: nginx-access

And here is logstash.conf

input {
    beats {
    port => 5044
  }
}

filter {
  grok {
    match => [ "message" , "%{COMBINEDAPACHELOG}+%{GREEDYDATA:extra_fields}"]
    overwrite => [ "message" ]
  }

  mutate {
            convert => ["response", "integer"]
            convert => ["bytes", "integer"]
            convert => ["responsetime", "float"]
    }

    geoip {
            source => "clientip"
            target => "geoip"
            add_tag => [ "nginx-geoip" ]
    }

    mutate {
            convert => [ "[geoip][coordinates]", "float"]
    } 

    date {
            match => [ "timestamp" , "dd/MMM/YYYY:HH:mm:ss Z" ]
            remove_field => [ "timestamp" ]
    }
    
    useragent {
            source => "agent"
    }
}

output {

stdout { codec => rubydebug }
if [type] == "nginx-access" {
  elasticsearch {
  hosts => localhost
  index => "nginx-access-%{+YYYY.MM.dd}"
}

}

Also on checking index template for nginx-access I see the geoip.location data type being float

{"nginx-access-2017.05.09":{"aliases":{},"mappings":{"nginx-access":{"properties":{"@timestamp":{"type":"date"},"@version":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"agent":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"auth":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"beat":{"properties":{"hostname":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"name":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"version":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}}}},"bytes":{"type":"long"},"clientip":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"device":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}}

,"geoip":{"properties":{"continent_code":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"country_code2":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"country_code3":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"country_name":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"ip":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"latitude":{"type":"float"},"location":{"type":"float"},"longitude":{"type":"float"}

So do i need to define a template to convert geoip.location field type to geopoint ? Or can someone lemme know how do i fix this ?

2

https://www.elastic.co/blog/geoip-in-the-elastic-stack has some good resources to assist.

1 Like
3

I went through the resource. Also tried overriding the default template
with the config like mentioned below

template => "/filebeat-index-template.json"
template_overwrite => true

Here is the filebeat-index-template.json file

{
"mappings": {
"default": {
"_all": {
"enabled": true,
"norms": {
"enabled": false
}
},
"dynamic_templates": [
{
"template1": {
"mapping": {
"doc_values": true,
"ignore_above": 1024,
"index": "not_analyzed",
"type": "{dynamic_type}"
},
"match": ""
}
}
],
"properties": {
"@timestamp": {
"type": "date"
},
"message": {
"type": "string",
"index": "analyzed"
},
"offset": {
"type": "long",
"doc_values": "true"
},
"geoip" : {
"type" : "object",
"dynamic": true,
"properties" : {
"location" : { "type" : "geo_point" }
}
}
}
}
},
"settings": {
"index.refresh_interval": "5s"
},
"template": "filebeat-"
}

Reference: https://github.com/elastic/logstash/issues/2952

Still I am getting geoip.location saved as number instead of geopoint in the new index created. Am i doing something wrong here ?

4

Is that the actual path?

5

Yes i mentioned the correct path for filebeat-index-template.json in template.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.