I've used a range of resources to try and figure this out. The most useful has been:
- A great guide but the commands don't work with version 7.x: http://www.sanjeevnandam.com/blog/logstash-convert-zipcodepostal-code-to-geo_point-latitudelongitude
- Also a great blog. The solution uses GeoIP, an approach I tried to take initially but couldn't get it to work: https://www.elastic.co/blog/geoip-in-the-elastic-stack
- I have used a number of the elastic command references, such as https://www.elastic.co/guide/en/logstash/7.6/plugins-filters-geoip.html etc.
- I've read so many discussion posts I've not kept track of, some of which were really useful i.e: ELK 5.2.1 - still can't get geoip working. Many of the issues, as far as I can tell, are related to GeoIP rather than generic geo (lat/long) issues.
- Some sites showed me what was possible but without providing step-by-step guides: https://medium.com/@andrea.reon/elasticsearch-why-and-how-c79a43288a4b
- Other blogs looked like they were going to be really useful but are addressing a slightly different problem: http://www.tapsw.com/software-development-elasticsearch-uk-postcodes.php
I've not managed to find an example of someone creating a mapping for ES 7.x. The resources above mostly show mappings that use types and the _default_ parameter, which are now obsolete. It likely that my google-fu is rusty, so apologies if anyone reading this has written an article using 7.x, I've just not managed to find it yet.
I have spent a long time looking at Logstash as I thought this is where the problem lay. Now that the latitude/longitude data is available in discover, I wonder whether the issue is related to the index mapping or my understanding of how to generate a geo_point from Logstash.
My Elasticsearch mapping, if I've understood correctly what a mapping is, is as per my first post:
{
"settings" : {
"index" : {
"number_of_shards" : "1",
"refresh_interval" : "5s"
}
},
"index_patterns": [ "cm_delivery_locations" ],
"mappings" : {
"properties" : {
"delivery_postcode": { "type": "text" },
"delivery_date": { "type": "date" },
"delivery_location" : {
"dynamic" : true,
"properties" : {
"location" : { "type" : "geo_point" },
"latitude" : { "type" : "half_float" },
"longitude" : { "type" : "half_float" }
}
}
}
}
}
If I perform a document layer in Maps, Inspect shows the following request:
{
"docvalue_fields": [
"delivery_location.location"
],
"size": 10000,
"_source": false,
"stored_fields": [
"delivery_location.location"
],
"script_fields": {},
"query": {
"bool": {
"must": [],
"filter": [
{
"match_all": {}
},
{
"range": {
"delivery_date": {
"gte": "2017-01-01T00:00:00.000Z",
"lte": "2019-12-31T23:30:00.000Z",
"format": "strict_date_optional_time"
}
}
}
],
"should": [],
"must_not": []
}
}
}
And this response:
{
"took": 38,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 772,
"max_score": 0,
"hits": [
{
"_index": "cm_delivery_locations",
"_type": "_doc",
"_id": "WeDtuXABDb_U0HpUp96o",
"_score": 0
},
{
"_index": "cm_delivery_locations",
"_type": "_doc",
"_id": "SODtuXABDb_U0HpUp96o",
"_score": 0
},
{
... cut data ...
}
]
}
}
There are no dots shown on the roadmap. Does this mean that the geo_point coordinates are not being returned correctly? Should I expect to see Lat/Long data in the hits array?
Thanks,