# Problem converting latitude and longitude into a geo point for Kibana

**URL:** <https://discuss.elastic.co/t/problem-converting-latitude-and-longitude-into-a-geo-point-for-kibana/52856>\
**Category:** Logstash\
**Created:** [June 15, 2016, 11:41am UTC](https://discuss.elastic.co/t/problem-converting-latitude-and-longitude-into-a-geo-point-for-kibana/52856 "2016-06-15T11:41:21Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![storri](https://avatars.discourse-cdn.com/v4/letter/s/a9adbd/32.png) [@storri](https://discuss.elastic.co/u/storri)\
**Post date:** [June 15, 2016, 11:41am UTC](https://discuss.elastic.co/t/problem-converting-latitude-and-longitude-into-a-geo-point-for-kibana/52856/1 "2016-06-15T11:41:21Z")

</div>

I am having difficulty getting Logstash (2.2.4) to convert two parsed values, longitude and latitude, into a geo point for Kibana. The documentation for Elasticsearch presents an example for Geo Points where you make some sort of configuration file ([https://www.elastic.co/guide/en/elasticsearch/guide/current/geopoints.html](https://www.elastic.co/guide/en/elasticsearch/guide/current/geopoints.html)). I do no know:

1. Where does it go?
2. Is it overriding the elasticsearch template used by Logstash?
3. Is there a pre-existing filter plugin for Logstash that can take an latitude and longitude to produce a geo point? The only thing that I am aware of is geoip. The assumption of geoip is that the IP is fixed to a set latitude and longitude. In my case the IP is mobile with a new position every few seconds.

My Logstash is as follows:

```
input {
  file {
    path => "/opt/project/mylog.txt"
  }
}

filter {
  grok {
    add_tag => ["project", "message1"]
    match => { "message" => "%{DATE_US:date} %{TIME:time} message1: name:%{DATA:name}, lat:%{DATA:latitude}, lon:%{DATA:longitude}, alt:%{DATA:altitude}" }
  }

  grok {
    add_tag => ["project", "meessage2"]
    match => { "message" => "%{DATE_US:date} %{TIME:time} meessage2: name:%{DATA:name}, lat:%{DATA:latitude}, lon:%{DATA:longitude}, alt:%{DATA:altitude}, delta:%{DATA:delta}, status:%{WORD:status}" }
  }

 if "message1" in [tags]
 {
   mutate {
     add_field => { "[location][lat]" => "%{latitude}"
                    "[location][lon]" => "%{longitude}"
     }
   }

   mutate {
      convert => {
        "[location][lat]" => "float"
        "[location][lon]" => "float"
     }
   }
 }

 mutate {
    convert =>{
                "latitude" => "float"
                "longitude" => "float"
                "altitude" => "float"
                "delta" => "float"
        }
   }
}

output {
    if "message1" in [tags] and "project" in [tags]
    {
      elasticsearch
      {
             index => "project-%{+YYYY.MM.dd}"
             manage_template => "false"
             template => "/etc/logstash/templates/project-elasticsearch.json"
      }

      file {
         path => "/opt/project/message1.txt"
      }
    }
    else if "message2" in [tags] and "project" in [tags]
    {
      elasticsearch
      {
             index => "project-%{+YYYY.MM.dd}"
      }

      file {
         path => "/opt/project/message2.txt"
      }

    }
}

```

Now here is the template I copied from the logstash elasticsearch output plugin and modified to try an tell logstash to modify the "location" object to be considered as a geo point.

```
{
  "template" : "thunderstorm-*",
  "settings" : {
    "index.refresh_interval" : "5s"
  },
  "mappings" : {
    "_default_" : {
      "_all" : {"enabled" : true, "omit_norms" : true},
      "dynamic_templates" : [ {
        "message_field" : {
          "match" : "message",
          "match_mapping_type" : "string",
          "mapping" : {
            "type" : "string", "index" : "analyzed", "omit_norms" : true,
            "fielddata" : { "format" : "disabled" }
          }
        }
      }, {
        "string_fields" : {
          "match" : "*",
          "match_mapping_type" : "string",
          "mapping" : {
            "type" : "string", "index" : "analyzed", "omit_norms" : true,
            "fielddata" : { "format" : "disabled" },
            "fields" : {
              "raw" : {"type": "string", "index" : "not_analyzed", "ignore_above" : 256}
            }
          }
        }
      } ],
      "properties" : {
        "@timestamp": { "type": "date" },
        "@version": { "type": "string", "index": "not_analyzed" },
        "geoip" : {
          "dynamic": true,
          "properties" : {
            "ip": { "type": "ip" },
            "location" : { "type" : "geo_point" },
            "latitude" : { "type" : "float" },
            "longitude" : { "type" : "float" }
          }
        },
	"location": {
	    "type": "geo_point"
        }
      }
    }
  }
}

```

I would appreciate any insight into this problem. As well I would appreciate any suggestions for improvement since I relative new to Logstash/Elasticsearch/Kibana.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [June 15, 2016, 10:21pm UTC](https://discuss.elastic.co/t/problem-converting-latitude-and-longitude-into-a-geo-point-for-kibana/52856/2 "2016-06-15T22:21:12Z")

</div>

You need a single field with lat and lon in it, not a nested field. eg

```auto
mutate {
  add_field => ["[geoip][location]", "%{longitude}" ]
  add_field => ["[geoip][location]", "%{latitude}" ]
}

```

---

<div class="post-metadata">

**Author:** ![storri](https://avatars.discourse-cdn.com/v4/letter/s/a9adbd/32.png) [@storri](https://discuss.elastic.co/u/storri)\
**Post date:** [June 16, 2016, 12:04pm UTC](https://discuss.elastic.co/t/problem-converting-latitude-and-longitude-into-a-geo-point-for-kibana/52856/3 "2016-06-16T12:04:08Z")

</div>

I got an error about duplicate keys from using the suggestion exactly as it is written. I think what you are expressing is not an exact syntax but merely representative of a single field for each. I change my configuration to be:

```
if "position" in [tags]
{
   mutate {
     add_field => { "[geoip][latitude]" => "%{latitude}" }
     add_field => { "[geoip][longitude]" => "%{longitude}" }
   }

   mutate {
     convert => {
       "[geoip][latitude]" => "float"
       "[geoip][longitude]" => "float"
    }
   }
 }

```

Is more in line with what with your suggestion?

---

<div class="post-metadata">

**Author:** ![storri](https://avatars.discourse-cdn.com/v4/letter/s/a9adbd/32.png) [@storri](https://discuss.elastic.co/u/storri)\
**Post date:** [June 16, 2016, 1:02pm UTC](https://discuss.elastic.co/t/problem-converting-latitude-and-longitude-into-a-geo-point-for-kibana/52856/4 "2016-06-16T13:02:45Z")

</div>

Changing the output to what I posted today still does not make the latitude and longitude into a geo point when I view the data in Kibana.

Here is what Kibana sees for the JSON:

```
{
  "_index": "project-2016.06.16",
  "_type": "logs",
  "_id": "AVVagllAqFBA57QBrgLU",
  "_score": null,
  "_source": {
    "message": "06/16/2016 14:38:58 Position: name:SampleDFDevice, lat:34.06691243701163, lon:-81.15418291973535, alt:1000.0",
    "@version": "1",
    "@timestamp": "2016-06-16T18:38:59.299Z",
    "path": "/opt/system/mylog.txt",
    "host": "localhost.localdomain",
    "date": "06/16/2016",
    "time": "14:38:58",
    "name": "SampleDFDevice",
    "latitude": 34.06691243701163,
    "longitude": -81.15418291973535,
    "tags": [
      "project",
      "position",
      "_grokparsefailure"
    ],
    "location": {
      "latitude": 34.06691243701163,
      "longitude": -81.15418291973535
    }
  },
  "fields": {
    "@timestamp": [
      1466102339299
    ]
  },
  "sort": [
    1466102339299
  ]
}

```

I do not see how location is considered a geo point. Should there be another entry in location for making it a geo point?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [June 16, 2016, 8:57pm UTC](https://discuss.elastic.co/t/problem-converting-latitude-and-longitude-into-a-geo-point-for-kibana/52856/5 "2016-06-16T20:57:56Z")

</div>

Have a read of [https://www.elastic.co/guide/en/elasticsearch/reference/2.3/geo-point.html](https://www.elastic.co/guide/en/elasticsearch/reference/2.3/geo-point.html)

---

<div class="post-metadata">

**Author:** ![storri](https://avatars.discourse-cdn.com/v4/letter/s/a9adbd/32.png) [@storri](https://discuss.elastic.co/u/storri)\
**Post date:** [June 17, 2016, 5:33pm UTC](https://discuss.elastic.co/t/problem-converting-latitude-and-longitude-into-a-geo-point-for-kibana/52856/6 "2016-06-17T17:33:48Z")

</div>

I see the script that I can use to set the mapping. I put the following into /etc/elasticsearch/templates in a file called location.json:

```
{
    "location_mapping": {
        "my_type": {
            "properties": {
                "location": {
                    "type": "geo_point"
                }
            }
        }
    }
}

```

Was this the direction you are suggesting?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [June 17, 2016, 10:07pm UTC](https://discuss.elastic.co/t/problem-converting-latitude-and-longitude-into-a-geo-point-for-kibana/52856/7 "2016-06-17T22:07:05Z")

</div>

As per the docs, your naming is wrong;

> Geo-point expressed as an object, with lat and lon keys.

```auto
"location": {
      "lat": 34.06691243701163,
      "lon": -81.15418291973535
    }

```

---

<div class="post-metadata">

**Author:** ![storri](https://avatars.discourse-cdn.com/v4/letter/s/a9adbd/32.png) [@storri](https://discuss.elastic.co/u/storri)\
**Post date:** [June 20, 2016, 1:05pm UTC](https://discuss.elastic.co/t/problem-converting-latitude-and-longitude-into-a-geo-point-for-kibana/52856/8 "2016-06-20T13:05:12Z")

</div>

Mark,

I appreciate your patience and perseverance in helping me with the problem. I have re-read the geo point documentation page and see the line you have quoted. I see that it gives an example JSON that describes an geo point object as:

```
PUT my_index/my_type/1
{
  "text": "Geo-point as an object",
  "location": { 
    "lat": 41.12,
    "lon": -71.34
  }
}

```

I changed the logstash file to read:

```
input { 
    file {
         path => "/opt/event/mylog.txt"
    }
}

filter {
    grok {
        add_tag => ["myproject", "position"]
        match => { "message" => "%{DATE_US:date} %{TIME:time} Position: name:%{DATA:name}, lat:%{DATA:latitude}, lon:%{DATA:longitude}, alt:%{DATA:altitude}" }
    }

    if "position" in [tags]
    {
       mutate {
         add_field => { "[location][lat]" => "%{latitude}" }
         add_field => { "[location][lon]" => "%{longitude}" }
       }

       mutate {
         convert => {
           "[location][lat]" => "float"
           "[location][lon]" => "float" 
        }
       }
   }

    mutate {
           convert =>{
                   "latitude" => "float"
                   "longitude" => "float"
                   "altitude" => "float"
                   "delta" => "float"
           }
    }
}

output { 
       if "position" in [tags] and "myproject" in [tags] 
       {

         elasticsearch
         {
                index => "myproject-%{+YYYY.MM.dd}"
               manage_template => "false"
               template => "/etc/logstash/templates/myproject-elasticsearch.json"
         }

         file {
            path => "/opt/cyberquest/position.txt"
         }

       }
       else if "heartbeat" in [tags] and "thunderstorm" in [tags]
       {

         elasticsearch
         {
                index => "myproject-%{+YYYY.MM.dd}"
         }

         file {
            path => "/opt/cyberquest/heartbeat.txt"
         }

       }
}

```

Which resulted in the following JSON object for a position:

```
{
  "_index": "myproject-2016.06.20",
  "_type": "logs",
  "_id": "AVVt60EVrNq0NqkairlO",
  "_score": null,
  "_source": {
    "message": "06/20/2016 09:06:21 Position: name:SampleDFDevice, lat:34.11946855074746, lon:-81.92403913044855, alt:1000.0",
    "@version": "1",
    "@timestamp": "2016-06-20T13:06:21.507Z",
    "path": "/opt/event/mylog.txt",
    "host": "localhost.localdomain",
    "date": "06/20/2016",
    "time": "09:06:21",
    "name": "SampleDFDevice",
    "latitude": 34.11946855074746,
    "longitude": -81.92403913044855,
    "tags": [
      "myproject",
      "position",
      "_grokparsefailure"
    ],
    "location": {
      "lat": 34.11946855074746,
      "lon": -81.92403913044855
    }
  },
  "fields": {
    "@timestamp": [
      1466427981507
    ]
  },
  "sort": [
    1466427981507
  ]
}

```

using the elasticsearch template of:

```
    {
        "location_mapping": {
            "my_type": {
                "properties": {
                    "location": {
                        "type": "geo_point",
                        "geohash_prefix": true
                    }
                }
            }
        }
    }

```

When I go into Kibana and do:

1. Click on Visualize
2. Select the myproject from the search source
3. Click on Geo Coordinates
4. Select Geo Coordinates as the bucket type

I see Aggregation say "Geohash" and below that I see a error message:

> No Compatible Fields: The "myproject-\*" index pattern does not contain any of the following field types: geo\_point.

This is why I have been posting here. I want to be able to plot the geo points on a map.

---

<div class="post-metadata">

**Author:** ![storri](https://avatars.discourse-cdn.com/v4/letter/s/a9adbd/32.png) [@storri](https://discuss.elastic.co/u/storri)\
**Post date:** [June 20, 2016, 6:19pm UTC](https://discuss.elastic.co/t/problem-converting-latitude-and-longitude-into-a-geo-point-for-kibana/52856/9 "2016-06-20T18:19:50Z")

</div>

Right I am not sure my template is being loaded and matching effectively. How do you double check a template is properly formatted and it is matching?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [June 22, 2016, 12:42am UTC](https://discuss.elastic.co/t/problem-converting-latitude-and-longitude-into-a-geo-point-for-kibana/52856/10 "2016-06-22T00:42:17Z")

</div>

Is it the template above?

---

<div class="post-metadata">

**Author:** ![storri](https://avatars.discourse-cdn.com/v4/letter/s/a9adbd/32.png) [@storri](https://discuss.elastic.co/u/storri)\
**Post date:** [June 22, 2016, 11:31am UTC](https://discuss.elastic.co/t/problem-converting-latitude-and-longitude-into-a-geo-point-for-kibana/52856/11 "2016-06-22T11:31:10Z")

</div>

No. As I read again the documentation for logstash and read a forum ([Add Geopoint based off of parsed value to logstash config](https://discuss.elastic.co/t/add-geopoint-based-off-of-parsed-value-to-logstash-config/26580/3)) I copied the logstash elasticsearch output plugin template and attempted to modify it for my needs.

Here is my template:

```
{
  "template" : "myproject-*",
  "settings" : {
    "index.refresh_interval" : "5s"
  },
  "mappings" : {
    "_default_" : {
      "_all" : {"enabled" : true, "omit_norms" : true},
      "dynamic_templates" : [ {
        "message_field" : {
          "match" : "message",
          "match_mapping_type" : "string",
          "mapping" : {
            "type" : "string", "index" : "analyzed", "omit_norms" : true,
            "fielddata" : { "format" : "disabled" }
          }
        }
      }, {
        "string_fields" : {
          "match" : "*",
          "match_mapping_type" : "string",
          "mapping" : {
            "type" : "string", "index" : "analyzed", "omit_norms" : true,
            "fielddata" : { "format" : "disabled" },
            "fields" : {
              "raw" : {"type": "string", "index" : "not_analyzed", "ignore_above" : 256}
            }
          }
        }
      } ],
      "properties" : {
        "@timestamp": { "type": "date" },
        "@version": { "type": "string", "index": "not_analyzed" },
        "geoip" : {
          "dynamic": true,
          "properties" : {
            "ip": { "type": "ip" },
            "location" : { "type" : "geo_point" },
            "latitude" : { "type" : "float" },
            "longitude" : { "type" : "float" }
          }
        },
	"location" : { "type": "geo_point" }
      }
    }
  }
}

```

When I look at the geo point documentation ([https://www.elastic.co/guide/en/elasticsearch/reference/current/geo-point.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/geo-point.html)) is see that the json for the commands to set a geo point appear different than what I see in my output. I see:

```
"location": {
      "lat": 33.48414869262067,
      "lon": -81.8800662624854
    }

```

when the examples seem to have extra curly braces:

```
{
"location": {
      "lat": 33.48414869262067,
      "lon": -81.8800662624854
    }
} 

```

Do the extra braces matter?

---

<div class="post-metadata">

**Author:** ![iamthealex](https://avatars.discourse-cdn.com/v4/letter/i/e9c0ed/32.png) [@iamthealex](https://discuss.elastic.co/u/iamthealex)\
**Post date:** [October 14, 2016, 8:02pm UTC](https://discuss.elastic.co/t/problem-converting-latitude-and-longitude-into-a-geo-point-for-kibana/52856/12 "2016-10-14T20:02:25Z")

</div>

Did you succeed in getting this work?  
I'm about to try my hand at converting lat/lon information into geo\_point for Kibana visualization.

---

<div class="post-metadata">

**Author:** ![storri](https://avatars.discourse-cdn.com/v4/letter/s/a9adbd/32.png) [@storri](https://discuss.elastic.co/u/storri)\
**Post date:** [October 17, 2016, 2:26pm UTC](https://discuss.elastic.co/t/problem-converting-latitude-and-longitude-into-a-geo-point-for-kibana/52856/13 "2016-10-17T14:26:49Z")

</div>

We got something to work but I don't recall what we did. It has been two months since I looked at it.

---

<div class="post-metadata">

**Author:** ![iamthealex](https://avatars.discourse-cdn.com/v4/letter/i/e9c0ed/32.png) [@iamthealex](https://discuss.elastic.co/u/iamthealex)\
**Post date:** [October 17, 2016, 2:51pm UTC](https://discuss.elastic.co/t/problem-converting-latitude-and-longitude-into-a-geo-point-for-kibana/52856/14 "2016-10-17T14:51:46Z")

</div>

Thanks for checking back in  
I did get it to work.  
Works for me like this:

```
PUT /alex-locations-a/v1/1
{
  "text" : "a point",
  "@timestamp" : 1476415349950,
  "na-location" : {
    "lat" : 41.12,
    "lon" : -71.34
  }
}

PUT /alex-locations-a/v1/2
{
  "text" : "another point",
  "@timestamp" : 1476415349951,
  "na-location" : {
    "lat" : 42.12,
    "lon" : -71.34
  }
}

with this mapping:
PUT _template/alex-locations
{
  "template": "alex-locations*",
  "settings": {},
  "mappings": {
    "_default_": {
      "properties": {
        "na-location": {
          "type": "geo_point"
        },
        "@timestamp": {
          "format": "strict_date_optional_time||epoch_millis",
          "type": "date"
        }
      }
    }
  }
}
```

---

<div class="post-metadata">

**Author:** ![slinky](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/slinky/32/13323_2.png) [@slinky](https://discuss.elastic.co/u/slinky)\
**Post date:** [December 13, 2016, 3:58pm UTC](https://discuss.elastic.co/t/problem-converting-latitude-and-longitude-into-a-geo-point-for-kibana/52856/15 "2016-12-13T15:58:24Z")

</div>

@iamthealex I tried out your example with the Kibana Console and it works like a charm.  
Because of your post I understand now completely. Many thanks! 😁

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:29am UTC](https://discuss.elastic.co/t/problem-converting-latitude-and-longitude-into-a-geo-point-for-kibana/52856/16 "2017-07-06T04:29:54Z")

</div>