[geoip.location is not of type geo_point] Index pattern does not contain any of the following field types: geo_point


(Kshitij Karthick) #1

Problem

  • I am not able to plot any coordinate map as index pattern does not contain any of the following field types: geo_point.
  • Upon further investigating the index pattern - the geoip.location has properties [loc, lat] but on request from the elastic search for the mapping the following key is not of type geo_point.
  • I am utilising logstash "COMBINEDAPACHELOG" capability with geoip plugin to parse all the keys [ keys such as country_name, city_name are obtained from the same ]
  • I could see no geo_point type for geoip.location when i obtained the mapping from elastic search. [Find snippet below]

Current Environment

  • Stack being utilised is ELK with filebeat
  • Versions
    • Elastic Search - 5.5.0v
    • Logstash - 5.5.0v
    • Kibana - 5.5.0v
    • Filebeats - 5.5.1v

Log stash Pipeline

input {
	beats {
    	port => 5000
  	}
}
filter {
    grok {
        match => { "message" => "%{COMBINEDAPACHELOG}"}
    }
    geoip {
        source => "clientip"
    }
    date {
    	match => [ "timestamp", "dd/MMM/yyyy:HH:mm:ss Z" ]
    }
}
output {
	elasticsearch {
		hosts => "elasticsearch:9200"
		index => "apache-logs"
	}
}

GET Mapping from Elastic Search - Short Snipper from the actual [Type Mismatch here]

{
  "apache-logs": {
    "mappings": {
      "log": {
        "properties": {
          "@timestamp": {
            "type": "date"
          },
          "clientip": {
            "fields": {
              "keyword": {
                "ignore_above": 256,
                "type": "keyword"
              }
            },
            "type": "text"
          },
          "geoip": {
            "properties": {
              "city_name": {
                "fields": {
                  "keyword": {
                    "ignore_above": 256,
                    "type": "keyword"
                  }
                },
                "type": "text"
              },
              "ip": {
                "fields": {
                  "keyword": {
                    "ignore_above": 256,
                    "type": "keyword"
                  }
                },
                "type": "text"
              },
              "latitude": {
                "type": "float"
              },
              "location": {
                "properties": {
                  "lat": {
                    "type": "float"
                  },
                  "lon": {
                    "type": "float"
                  }
                }
              },
              "longitude": {
                "type": "float"
              }
            }
          }
        }
      }
    }
  }
}

Solutions tried

  • Refreshing the index multiple times.
  • The map rendering had worked previously with older versions of elastic search 4.xx, not able to replicate the same.
  • PUT mapping_index to set type geo_point explicitly which returned the following error
{
  "error": {
    "root_cause": [
      {
        "type": "action_request_validation_exception",
        "reason": "Validation Failed: 1: mapping type is missing;"
      }
    ],
    "type": "action_request_validation_exception",
    "reason": "Validation Failed: 1: mapping type is missing;"
  },
  "status": 400
}
  • Explicitly set geoip.location.lat, long to float as indicated from Digital Ocean Blog
  • Followed the steps exactly from Elastic blog but still did not resolve the issue

I would be grateful if anyone can help me resolve this issue.


How to update the type of a field
(Sam) #2

I too am having the same issue, using Elasticsearch 5.5.0, Logstash 5.5.0, Kibana 5.5.0 but using Filebeat v5.1.2


(Sam) #3

Ah ha! I've just resolved this. I was able to update the logstash template and force the mapping to be a geo_point. (I did this from the DevTools pane in Kibana)

PUT _template/logstash
{
  "template": "logstash-*", 
  "settings": {
    "number_of_replicas": 1,
    "number_of_shards": 2
  },
  "mappings": {
    "my_type": {
      "dynamic": "true",
      "properties": {
        "geoip": {
          "dynamic": true,
          "properties": {
            "location": {
              "type": "geo_point"
            }
          }
        }
      }
    }
  }
}

After doing this I deleted my index, which logstash then recreated with the correct mappings.

Best of luck!


(Kshitij Karthick) #4

Thank you for the information. Your solution worked for me :smiley:

Though while checking all the templates present

GET http://localhost:9200/_template
{
    apache-logs:  {	
        order
        template: "apache-logs"
        settings	
        mappings: Object
        aliases
    }
    logstash	{
        order: 0
        version: 50001
        template: "logstash-*"
        settings	
        index	
        refresh_interval: "5s"
        mappings: Object
        aliases
    }
}

Upon checking the default logstash template, it already had geoip.location with type "geo_point" without explicitly me setting it like in the case of my index apache-logs

So i purged all the data from logstash, filebeat, elastic-search & Kibana and started the whole process again with the only change being my index name as logstash-apache and the geoip.location by default in kibana had the geo_point type and everything worked as is.

Thanks for your input, maybe try the same and see if it works for you directly without the explicit update of the mapping.


(system) #5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.