AWS Version of Elasticsearch, issue with GeoIP with Lambda Function

Elastic Stack Elasticsearch
1

Hi All,

I'm very new to Elasticsearch and I'm trying to get it to load some data from our AWS application load balancer with some geoip data. Well to be precise, I haven't even gotten to the Lambda converting the ip to coordinates part yet, because I having issues with mapping a coordinates field in Elasticsearch. I've created a mapping template that will treat a 'coordinates' filed as a geo_point type which it shows in Kibana, but I get an error that says that:

"error": {
        "type": "mapper_parsing_exception",
        "reason": "failed to parse",
        "caused_by": {
            "type": "parse_exception",
            "reason": "geo_point expected"
        }
    }

Here is my mapping:

{
"cwl-2017.06.13": {
"mappings": {
  "coordinates": {
    "properties": {
      "coordinates": {
        "type": "geo_point"
      },
      "type": {
        "type": "text"
      }
    }
  }
}
}

Here is an example of the data I'm sending to Elasticsearch (I've removed data for brevity and identifying):

{
 "index": {
    "_index": "cwl-2017.06.13",
    "_type": "application-load-balancer-logs-1",
    "_id": "3339237098236054586560913755308454237516464130"
 }
}

{
 "received_bytes": 152,
 "target_group_arn": "arn:aws:...5f1e",
 "request": "GET http://www.example.com:80/ HTTP/1.1",
 "trace_id": "Root=1-593ffcc5-568e2ac6d7e052",
 "@id": "333923709862434308454237516464130",
 "@timestamp": "2017-06-13T14:55:01.862Z",
 "@log_group": "application-load-balancer-logs-1",
 "@log_stream": "application-load-balancer-stream",
 "coordinates": [
    34,
    -82
 ]
}

I have tried that hardcoding that coordinates part a million different ways and have been banging my head on my desk trying to get this going, please any help you can provide I would greatly appreciate...

Thanks!!!

2

I seemed to have figured this out with the help of the blog post here:
https://jackhanington.com/blog/2014/12/11/create-a-custom-elasticsearch-template/

I changed my template to this:

{
 "template" : "cwl*",
"mappings" : {
  "application-load-balancer-logs-1" : {
     "properties": {
        "geoip":{
           "properties":{
              "ip":{"type":"ip"},
              "latitude":{"type":"double"},
              "location":{"type":"geo_point"},
              "longitude":{"type":"double"}
              }
        }
  }
}
}
}

Hopefully this helps someone else!

3

https://www.elastic.co/blog/geoip-in-the-elastic-stack may also be useful.

4

This is good, thank you!!

As a final point to this; anyone using AWS Elasticsearch who are interested in using the tile map functionality please note:

Using Elasticsearch with Kibana from AWS violates a term of service in regards to using the maps; so when you try it map will be blank and in the console the tiles images are pointed to example.com/path/to/image.png. Even though you can change the map service provider in the options in Kibana, as of now it will still show as example.com and will result in blank map tiles

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.