Geo_point issues

Elastic Stack Elasticsearch
1

HI all,

We are having major issues getting geolocation mapping working after upgrading to Elasticsearch 6.0 anf Kibana 6.0.

We have tried all sorts of things to try and set the location field to type:geo_point, but nothing seems to work.

We were using Elasticsearch 2.4.4 and were using the following index template:

$ curl -XGET '127.0.0.1:9200/_template?pretty'
{
  "providence_template" : {
    "order" : 2,
    "template" : "providence_doc-*",
    "settings" : { },
    "mappings" : {
      "post" : {
        "properties" : {
          "sourceid" : {
            "type" : "long"
          },
          "postidentifier" : {
            "index" : "not_analyzed",
            "type" : "string"
          },
          "link" : {
            "index" : "not_analyzed",
            "type" : "string"
          },
          "postdate" : {
            "type" : "date"
          },
          "sourcename" : {
            "index" : "not_analyzed",
            "type" : "string"
          },
          "title" : {
            "type" : "string"
          },
          "srid" : {
            "type" : "long"
          },
          "imageUrls" : {
            "index" : "not_analyzed",
            "type" : "string"
          },
          "location" : {
            "type" : "geo_point"
          },
          "text" : {
            "type" : "string"
          },
          "keyword" : {
            "index" : "not_analyzed",
            "type" : "string"
          },
          "entity" : {
            "index" : "not_analyzed",
            "type" : "string"
          },
          "group" : {
            "index" : "not_analyzed",
            "type" : "string"
          },
          "username" : {
            "index" : "not_analyzed",
            "type" : "string"
          }
        }
      }
    },
    "aliases" : { }
  }
}

The corresponding part of the index pattern related to the upgrade to 6.0 looks like:

}
            }
          },
          "location" : {
            "properties" : {
              "lat" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              },
              "lon" : {
                "type" : "text",
                "fields" : {
                  "keyword" : {
                    "type" : "keyword",
                    "ignore_above" : 256
                  }
                }
              }
            }

Unfortunately our only Elasticsearch developer who set things up is now on leave for a month and we are a bit stuck. One of our other devlopers managed to figure out that the index on the older 2.4.4 version was stored as a template. So he created a template in the new 6.0 version and it now looks like this:

curl -XPUT '127.0.0.1:9200/_template/providence_template?pretty' -H 'Content-Type: application/json' -d'
{
    "index_patterns": ["providence_doc-"],
    "template" : "providence_doc-*",
    "settings" : { },
    "mappings" : {
      "post" : {
        "properties" : {
          "sourceid" : {
            "type" : "long"
          },
          "postdate" : {
            "type" : "date"
          },
          "srid" : {
            "type" : "long"
          },
          "location" : {
            "type" : "geo_point"
          }
        }
      }
    },
    "aliases" : { }
}

He then re-booted everything. However, nothing seems to have changed. I have no fields that are geo_point to enable us to map geolocations.

Any advice would be HUGELY appreciated.

Thanks in advance.

Cheers,
Eric

2

Please format your code using </> icon as explained in this guide. It will make your post more readable.

Or use markdown style like:

```
CODE
```

What is the current mapping of your index?

GET providence_doc-XXXX/_mapping

replace XXXX with the right values.

3

Hi David,

Thanks for your response, I will try and go back and fix up my post.

The mapping of the index I just pulled using the Kibana Dev Tool is below. the location field is present but it does not appear that the geo_point field setting came across from the template:

CODE

{
"providence_doc-2017-12": {
"mappings": {
"post": {
"properties": {
"entity": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"group": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"imageUrl": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"keyword": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"link": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"location": {
"properties": {
"lat": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"lon": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"postdate": {
"type": "date"
},
"postidentifier": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"sourceid": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"sourcename": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"srid": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"text": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"title": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"username": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
},
"providence_doc-2017-11": {
"mappings": {
"post": {
"properties": {
"entity": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"group": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"imageUrl": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"keyword": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"link": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"location": {
"properties": {
"lat": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"lon": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"postdate": {
"type": "date"
},
"postidentifier": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"sourceid": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"sourcename": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"srid": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"text": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"title": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"username": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}
}

CODE
4

As you can see the formatting is still not correct. You can see that in the preview window anytime you write a response.

So your template does not seem to be applied here.
Actually you need to update your template as it does not conform I believe to elasticsearch 6.0.

For example string must be changed by text.
I guess you have some errors in your logs telling you something like this?

5

Hi David,

Apologies, still trying to figure out how to format the code here. Will give it another shot.

I am not a developer (I am a Director of a company ), I have a guy who has never worked with Elasticsearch trying to sort this issue out for us, so I am just trying to support him as best I can with research etc.

He will be able to check the logs for error messages tomorrow.

Are you saying that the last mapping I posted is not 6.0 compliant? Or the template we tried to use?

6

Or the template we tried to use?

Yes the template does not look compliant to me.

7

Thanks David,

We have updated the template. The mapping definitin we are using is:

  },
  "location" : {
    "properties" : {
      "lat" : {
        "type" : "text",
        "fields" : {
          "keyword" : {
            "type" : "keyword",
            "ignore_above" : 256
          }
        }
      },
      "lon" : {
        "type" : "text",
        "fields" : {
          "keyword" : {
            "type" : "keyword",
            "ignore_above" : 256
          }
        }
      }
    }
  },

However we are getting the following error:

"error" : {
    "root_cause" : [
      {
        "type" : "mapper_parsing_exception",
        "reason" : "Mapping definition for [location] has unsupported parameters:  [properties : {lon={type=keyword, fields={keyword={ignore_above=256, type=keyword}}}, lat={type=keyword, fields={keyword={ignore_above=256, type=keyword}}}}]"
      }
    ],
    "type" : "mapper_parsing_exception",
    "reason" : "Failed to parse mapping [post]: Mapping definition for [location] has unsupported parameters:  [properties : {lon={type=keyword, fields={keyword={ignore_above=256, type=keyword}}}, lat={type=keyword, fields={keyword={ignore_above=256, type=keyword}}}}]",
    "caused_by" : {
      "type" : "mapper_parsing_exception",
      "reason" : "Mapping definition for [location] has unsupported parameters:  [properties : {lon={type=keyword, fields={keyword={ignore_above=256, type=keyword}}}, lat={type=keyword, fields={keyword={ignore_above=256, type=keyword}}}}]"
    }
  },
  "status" : 400

This is occurring when he is trying to load the new template.

Any thoughts?

8

We managed to get it all sorted.

We removed the lat and lon definitions from indexes and made the location similar to the other sections.

However the removal of the use of string as a field type has create issues with being able to display URL's as images using the below settings:

image
image.png1444×402 14.7 KB

I now can't work out how to do this. We just tried making the field type 'text' but it had no effect.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.