Geo_point issues

EricWS · December 2, 2017, 8:09am

HI all,

We are having major issues getting geolocation mapping working after upgrading to Elasticsearch 6.0 anf Kibana 6.0.

We have tried all sorts of things to try and set the location field to type:geo_point, but nothing seems to work.

We were using Elasticsearch 2.4.4 and were using the following index template:

$ curl -XGET '127.0.0.1:9200/_template?pretty'
{
  "providence_template" : {
    "order" : 2,
    "template" : "providence_doc-*",
    "settings" : { },
    "mappings" : {
      "post" : {
        "properties" : {
          "sourceid" : {
            "type" : "long"
          },
          "postidentifier" : {
            "index" : "not_analyzed",
            "type" : "string"
          },
          "link" : {
            "index" : "not_analyzed",
            "type" : "string"
          },
          "postdate" : {
            "type" : "date"
          },
          "sourcename" : {
            "index" : "not_analyzed",
            "type" : "string"
          },
          "title" : {
            "type" : "string"
          },
          "srid" : {
            "type" : "long"
          },
          "imageUrls" : {
            "index" : "not_analyzed",
            "type" : "string"
          },
          "location" : {
            "type" : "geo_point"
          },
          "text" : {
            "type" : "string"
          },
          "keyword" : {
            "index" : "not_analyzed",
            "type" : "string"
          },
          "entity" : {
            "index" : "not_analyzed",
            "type" : "string"
          },
          "group" : {
            "index" : "not_analyzed",
            "type" : "string"
          },
          "username" : {
            "index" : "not_analyzed",
            "type" : "string"
          }
        }
      }
    },
    "aliases" : { }
  }
}

The corresponding part of the index pattern related to the upgrade to 6.0 looks like:

}
            }
          },
          "location" : {
            "properties" : {
              "lat" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "lon" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              }
            }

Unfortunately our only Elasticsearch developer who set things up is now on leave for a month and we are a bit stuck. One of our other devlopers managed to figure out that the index on the older 2.4.4 version was stored as a template. So he created a template in the new 6.0 version and it now looks like this:

curl -XPUT '127.0.0.1:9200/_template/providence_template?pretty' -H 'Content-Type: application/json' -d'
{
    "index_patterns": ["providence_doc-"],
    "template" : "providence_doc-*",
    "settings" : { },
    "mappings" : {
      "post" : {
        "properties" : {
          "sourceid" : {
            "type" : "long"
          },
          "postdate" : {
            "type" : "date"
          },
          "srid" : {
            "type" : "long"
          },
          "location" : {
            "type" : "geo_point"
          }
        }
      }
    },
    "aliases" : { }
}

He then re-booted everything. However, nothing seems to have changed. I have no fields that are geo_point to enable us to map geolocations.

Any advice would be HUGELY appreciated.

Thanks in advance.

Cheers,
Eric

dadoonet · December 2, 2017, 8:19am

Please format your code using </> icon as explained in this guide. It will make your post more readable.

Or use markdown style like:

```
CODE
```

What is the current mapping of your index?

GET providence_doc-XXXX/_mapping

replace XXXX with the right values.

EricWS · December 2, 2017, 8:40am

Hi David,

Thanks for your response, I will try and go back and fix up my post.

The mapping of the index I just pulled using the Kibana Dev Tool is below. the location field is present but it does not appear that the geo_point field setting came across from the template:

CODE

{
"providence_doc-2017-12": {
"mappings": {
"post": {
"properties": {
"entity": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"group": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"imageUrl": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"keyword": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"link": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"location": {
"properties": {
"lat": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"lon": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"postdate": {
"type": "date"
},
"postidentifier": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"sourceid": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"sourcename": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"srid": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"text": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"title": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"username": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
},
"providence_doc-2017-11": {
"mappings": {
"post": {
"properties": {
"entity": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"group": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"imageUrl": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"keyword": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"link": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"location": {
"properties": {
"lat": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"lon": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"postdate": {
"type": "date"
},
"postidentifier": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"sourceid": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"sourcename": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"srid": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"text": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"title": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"username": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}
}

CODE

dadoonet · December 2, 2017, 9:56am

As you can see the formatting is still not correct. You can see that in the preview window anytime you write a response.

So your template does not seem to be applied here.
Actually you need to update your template as it does not conform I believe to elasticsearch 6.0.

For example string must be changed by text.
I guess you have some errors in your logs telling you something like this?

EricWS · December 2, 2017, 10:10am

Hi David,

Apologies, still trying to figure out how to format the code here. Will give it another shot.

I am not a developer (I am a Director of a company ), I have a guy who has never worked with Elasticsearch trying to sort this issue out for us, so I am just trying to support him as best I can with research etc.

He will be able to check the logs for error messages tomorrow.

Are you saying that the last mapping I posted is not 6.0 compliant? Or the template we tried to use?

dadoonet · December 2, 2017, 4:06pm

Or the template we tried to use?

Yes the template does not look compliant to me.

EricWS · December 3, 2017, 10:20am

Thanks David,

We have updated the template. The mapping definitin we are using is:

  },
  "location" : {
    "properties" : {
      "lat" : {
        "type" : "text",
        "fields" : {
          "keyword" : {
            "type" : "keyword",
            "ignore_above" : 256
          }
        }
      },
      "lon" : {
        "type" : "text",
        "fields" : {
          "keyword" : {
            "type" : "keyword",
            "ignore_above" : 256
          }
        }
      }
    }
  },

However we are getting the following error:

"error" : {
    "root_cause" : [
      {
        "type" : "mapper_parsing_exception",
        "reason" : "Mapping definition for [location] has unsupported parameters:  [properties : {lon={type=keyword, fields={keyword={ignore_above=256, type=keyword}}}, lat={type=keyword, fields={keyword={ignore_above=256, type=keyword}}}}]"
      }
    ],
    "type" : "mapper_parsing_exception",
    "reason" : "Failed to parse mapping [post]: Mapping definition for [location] has unsupported parameters:  [properties : {lon={type=keyword, fields={keyword={ignore_above=256, type=keyword}}}, lat={type=keyword, fields={keyword={ignore_above=256, type=keyword}}}}]",
    "caused_by" : {
      "type" : "mapper_parsing_exception",
      "reason" : "Mapping definition for [location] has unsupported parameters:  [properties : {lon={type=keyword, fields={keyword={ignore_above=256, type=keyword}}}, lat={type=keyword, fields={keyword={ignore_above=256, type=keyword}}}}]"
    }
  },
  "status" : 400

This is occurring when he is trying to load the new template.

Any thoughts?

EricWS · December 3, 2017, 11:08pm

We managed to get it all sorted.

We removed the lat and lon definitions from indexes and made the location similar to the other sections.

However the removal of the use of string as a field type has create issues with being able to display URL's as images using the below settings:

I now can't work out how to do this. We just tried making the field type 'text' but it had no effect.

system · December 31, 2017, 11:08pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Unable to reference geo_point type field Elasticsearch	5	566	July 23, 2019
Geo_point & index_templates not working Elasticsearch	5	1438	October 16, 2017
Geo_point problems with multiple indexes and Logstash Elasticsearch	16	1306	October 16, 2019
Problems with geo_point Logstash	8	594	August 17, 2018
Elasticsearch 7.x and geo_point Type: can not do it via index_template mapping? Elasticsearch	3	802	November 19, 2019

Geo_point issues

Related topics