# Visualization of lat lon coordinates on Kibana's graph

**URL:** <https://discuss.elastic.co/t/visualization-of-lat-lon-coordinates-on-kibanas-graph/106177>\
**Category:** Kibana\
**Created:** [November 2, 2017, 11:13am UTC](https://discuss.elastic.co/t/visualization-of-lat-lon-coordinates-on-kibanas-graph/106177 "2017-11-02T11:13:47Z")\
**Posts on this page:** 19\
**Page:** 1

<div class="post-metadata">

**Author:** ![citanionarrei](https://avatars.discourse-cdn.com/v4/letter/c/da6949/32.png) [@citanionarrei](https://discuss.elastic.co/u/citanionarrei)\
**Post date:** [November 2, 2017, 11:13am UTC](https://discuss.elastic.co/t/visualization-of-lat-lon-coordinates-on-kibanas-graph/106177/1 "2017-11-02T11:13:47Z")

</div>

hi  
i have a problem  
i have log files with two fields: "lat" and "lon"

 ![Screenshot (8)](https://us1.discourse-cdn.com/elastic/original/3X/0/8/08c55aef32bc430dead0d2401dea26b40fe5beb7.png)  
and this error on kibana visualization

"the "meetup-\*" index pattern does not contain any of the following field types: geo\_point"

 ![Screenshot (9)](https://us1.discourse-cdn.com/elastic/original/3X/9/4/9498b8dc769c57d9fdcaf78ec04580d8af708438.png)

how can i resolve this problem?  
how i can set logstash for it  
Now i'm using this conf. on logstash:

```
input {file {
path => ["C:\Users\Lock\Desktop\meetup.json"]  
start_position => "beginning"
sincedb_path => "/dev/null"
exclude => "*.gz"
type => "log"
}}

filter
{
if [type] == "log"
{
grok {	
	match => ["message", "(?<meetup>{\".*\".*})"]
	break_on_match => true
	add_field => { "type" => "json_meetup"} 	
	}}
if "_grokparsefailure" in [tags]{drop {}}	
else {json {source => "meetup"}	}
}

output{ elasticsearch {
index => "meetup-%{+YYYY.MM.dd}"
document_type => "json_meetup"
codec => json_lines
}

```

thanks

---

<div class="post-metadata">

**Author:** ![mikelD](https://avatars.discourse-cdn.com/v4/letter/m/d26b3c/32.png) [@mikelD](https://discuss.elastic.co/u/mikelD)\
**Post date:** [November 2, 2017, 3:29pm UTC](https://discuss.elastic.co/t/visualization-of-lat-lon-coordinates-on-kibanas-graph/106177/2 "2017-11-02T15:29:03Z")

</div>

You have to define the field that contains the latitude and longitude as a geo\_point, in the index mapping, for reference in this link [https://www.elastic.co/guide/en/elasticsearch/reference/current/geo-point.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/geo-point.html) you have the different ways it can be specified.

You can define the field in a index template for the new indexes.

---

<div class="post-metadata">

**Author:** ![citanionarrei](https://avatars.discourse-cdn.com/v4/letter/c/da6949/32.png) [@citanionarrei](https://discuss.elastic.co/u/citanionarrei)\
**Post date:** [November 2, 2017, 3:33pm UTC](https://discuss.elastic.co/t/visualization-of-lat-lon-coordinates-on-kibanas-graph/106177/3 "2017-11-02T15:33:28Z")

</div>

Thank you for your answer...  
I am not yet very experienced  
and I did not understand how I can do this.  
Can you help me?  
where do I build the template? on logstash or on dev tool on kibana?

this is my json

 ![Screenshot (10)](https://us1.discourse-cdn.com/elastic/original/3X/6/2/62917e91d286a54a53c3833054445b3c8dce469b.png)

i'm using this as source of logs  
[http://stream.meetup.com/2/rsvps](http://stream.meetup.com/2/rsvps)

how can i create template for logstash and create field for geoip?  
this dont work

filter{  
grok {  
match =\> ["message", "(?{"._"._})"]  
break\_on\_match =\> true  
add\_field =\> { "type" =\> "json\_meetup"}   
}

mutate {  
add\_field =\> { "[venue][lat]" =\> "%{venue.lat}" }  
add\_field =\> { "[venue][lon]" =\> "%{venue.lon}" }  
}  
mutate {  
convert =\> {"[venue][lat]" =\> "float"}  
convert =\> {"[venue][lon]" =\> "float"}  
}

---

<div class="post-metadata">

**Author:** ![mikelD](https://avatars.discourse-cdn.com/v4/letter/m/d26b3c/32.png) [@mikelD](https://discuss.elastic.co/u/mikelD)\
**Post date:** [November 3, 2017, 9:15am UTC](https://discuss.elastic.co/t/visualization-of-lat-lon-coordinates-on-kibanas-graph/106177/4 "2017-11-03T09:15:21Z")

</div>

First, you need to create a template for your index that has a field mapped as geo\_point. You can do this with the Dev Tool on kibana,with this tool you can communicate with the elasticsearch node and make searches, insert data or configure parameters.

You can see the mapping of one of your index with this command:  
`GET [index_id]/_mapping`  
in my case  
`GET gps-2017.08.10/_mapping`

Using the mapping as a template, you can add a field(for example location) and map it as a geo\_point (It's better to add a new field for the coordinates because if you use one you already had mapped differently in previous index, elasticsearch will give you error becouse the same field is mapped differently in two related indexes):

> "gps-2017.08.10": {  
> "mappings": {  
> "logs": {  
> "properties": {  
> "@timestamp": {  
> "type": "date"  
> },  
> "@version": {  
> "type": "text",  
> "fields": {  
> "keyword": {  
> "type": "keyword",  
> "ignore\_above": 256  
> }  
> }  
> },  
> **"coordinates": {**  
> **"type": "geo\_point"**  
> **},**  
> "dev\_addr": {  
> "type": "text",  
> "fields": {  
> "keyword": {  
> "type": "keyword",  
> "ignore\_above": 256  
> }  
> }  
> },  
> "dev\_id": {  
> "type": "text",  
> "fields": {  
> "keyword": {  
> "type": "keyword",  
> "ignore\_above": 256  
> }  
> }  
> }  
> "host": {  
> "type": "ip"  
> }  
> }  
> }  
> }

Then you can use this mapping to create a template, like this one:

> {  
> "template": "gps-\*",  
> "order": 0,  
> "settings": {  
> "index.mapping.ignore\_malformed": true  
> },  
> "mappings": {  
> "logs": {  
> "properties": {  
> "coordinates": {  
> "type": "geo\_point"  
> },  
> "@timestamp": {  
> "type": "date"  
> },  
> "@version": {  
> "type": "text",  
> "fields": {  
> "keyword": {  
> "type": "keyword",  
> "ignore\_above": 256  
> }  
> }  
> }  
> "dev\_id": {  
> "type": "text",  
> "fields": {  
> "keyword": {  
> "type": "keyword",  
> "ignore\_above": 256  
> }  
> }  
> }  
> "host": {  
> "type": "ip"  
> },  
> "dev\_addr": {  
> "type": "text",  
> "fields": {  
> "keyword": {  
> "type": "keyword",  
> "ignore\_above": 256  
> }  
> }  
> },  
> "dev\_id": {  
> "type": "text",  
> "fields": {  
> "keyword": {  
> "type": "keyword",  
> "ignore\_above": 256  
> }  
> }  
> }  
> }  
> }  
> }  
> }

To add the new template to elasticsearch, you insert the next command in the dev tool followed by the template, the index identifier in my example would be 'gps':  
`PUT _template/[index_identifier]`  
` [Template]`

**Once you have a template with a geo\_point field defined** , you can insert data in this index, the field coordinates must be formatted like this:

- Geo-point as an object

> "coordinates": {  
> "lat": 41.12,  
> "lon": -71.34  
> }

- Geo-point as a string

> "coordinates": "41.12,-71.34"

- Geo-point as a geohash

> "coordinates": "drm3btev3e86"

- Geo-point as an array

> "coordinates": [-71.34, 41.12]

If you do this right and you have no conflict in the indexes mapping the maptiles pluging from kibana should work correctly.

---

<div class="post-metadata">

**Author:** ![citanionarrei](https://avatars.discourse-cdn.com/v4/letter/c/da6949/32.png) [@citanionarrei](https://discuss.elastic.co/u/citanionarrei)\
**Post date:** [November 3, 2017, 9:44am UTC](https://discuss.elastic.co/t/visualization-of-lat-lon-coordinates-on-kibanas-graph/106177/5 "2017-11-03T09:44:13Z")

</div>

ohhhh many thanks friend..  
ok i now i see it on my mapping

```
 "venue": {
        "properties": {
          "lat": {
            "type": "float"
          },
          "lon": {
            "type": "float"
          },

```

than ... i create an external template "meetup1.json" that i can load with output of logstash in this way and overwrite the defoult template  
output

```
{ 
elasticsearch {
index => "meetup"
document_type => "json_meetup"
manage_template => true
template => "C:\Users\Lock\Desktop\meetup1.json"
template_overwrite => "true"
codec => json_lines
}

```

Than if i change " float " with "geo\_point" can i resolve the problem on kibana?

must i add location field ?

---

<div class="post-metadata">

**Author:** ![mikelD](https://avatars.discourse-cdn.com/v4/letter/m/d26b3c/32.png) [@mikelD](https://discuss.elastic.co/u/mikelD)\
**Post date:** [November 3, 2017, 10:15am UTC](https://discuss.elastic.co/t/visualization-of-lat-lon-coordinates-on-kibanas-graph/106177/6 "2017-11-03T10:15:02Z")

</div>

you need to modify the lat and lon so they are inside another field wich is defined as geo\_point, because if you don't do it, elastic won't identify them as coordinates just numbers.

The data must have a format like this:

> "venue":{  
> "venue\_name":"Name",  
> ...  
> "coordinates":{  
> "lat":-42.7,  
> "long":-25.1  
> },  
> ...  
> }

You will need to parse your data source because with the data you are receiving you don't have an easy way to do it.

it's not necessary to add the location field on logstash, because you can format it to geo\_point, Logstash only knows standard data types. It is recommended to have the template in my experience to avoid problems with conflicting mappings in the indexes.

It happened to me that i was sending a data of type float but because the value of the field was exact (24.0) it mapped it as a long in the index, this kind of things creates a lot of problems.

---

<div class="post-metadata">

**Author:** ![citanionarrei](https://avatars.discourse-cdn.com/v4/letter/c/da6949/32.png) [@citanionarrei](https://discuss.elastic.co/u/citanionarrei)\
**Post date:** [November 4, 2017, 8:30am UTC](https://discuss.elastic.co/t/visualization-of-lat-lon-coordinates-on-kibanas-graph/106177/7 "2017-11-04T08:30:32Z")

</div>

> [@mikelD](#):
>
> Once you have a template with a geo\_point field defined, you can insert data in this index, the field coordinates must be formatted like this:
> 
> Geo-point as an object
> 
> "coordinates": {
> 
> "lat": 41.12,
> 
> "lon": -71.34
> 
> }
> 
> Geo-point as a string
> 
> "coordinates": "41.12,-71.34"
> 
> Geo-point as a geohash
> 
> "coordinates": "drm3btev3e86"
> 
> Geo-point as an array
> 
> "coordinates": [-71.34, 41.12]
> 
> If you do this right and you have no conflict in the indexes mapping the maptiles pluging from kibana should work correctly.

if my "coordinates" are formatted like these ways, can i use only the mapping "coordinates": {type:geo\_point} ?

---

<div class="post-metadata">

**Author:** ![citanionarrei](https://avatars.discourse-cdn.com/v4/letter/c/da6949/32.png) [@citanionarrei](https://discuss.elastic.co/u/citanionarrei)\
**Post date:** [November 4, 2017, 9:31am UTC](https://discuss.elastic.co/t/visualization-of-lat-lon-coordinates-on-kibanas-graph/106177/8 "2017-11-04T09:31:54Z")

</div>

ok now i have

 ![Screenshot (26)](https://us1.discourse-cdn.com/elastic/original/3X/1/9/1969d744382eaa65afab78e93fbfb436fc86030a.png)

fields"venue.lat" and " venue.lon" are now type: geo\_point

my template is

```
PUT _template/template_1
{
 "template":"meetup-*",
 "settings":{
"number_of_shards": 1
},

"mappings": {
  "json_meetup": {
    "properties": {
      "venue": {
        "properties": {
          "lat": {
            "type": "geo_point"
          },
          "lon": {
            "type": "geo_point"
          }
          
          
        } } }

```

but when i try to visualize my points i can choose only venue.lat OR venue.lon field than in this way it is impossible a visualization

 ![Screenshot (27)](https://us1.discourse-cdn.com/elastic/original/3X/1/d/1de07c86b210359582ca00d17c3493d5612f70d3.png)

---

<div class="post-metadata">

**Author:** ![mikelD](https://avatars.discourse-cdn.com/v4/letter/m/d26b3c/32.png) [@mikelD](https://discuss.elastic.co/u/mikelD)\
**Post date:** [November 6, 2017, 7:27am UTC](https://discuss.elastic.co/t/visualization-of-lat-lon-coordinates-on-kibanas-graph/106177/9 "2017-11-06T07:27:05Z")

</div>

That happens because you have defined both venue.lat and venue.long as a geo\_point, your geo\_point must be a unique field that contains the lat and lon.

![imagen](https://us1.discourse-cdn.com/elastic/original/3X/4/4/440e8d9cf6a5f0bd67f4406a3cf8651ce47f1436.png)

In this Image the field "location" is defined as a geo\_point, and the fields "lat" and "lon" are two subfields of location.

---

<div class="post-metadata">

**Author:** ![citanionarrei](https://avatars.discourse-cdn.com/v4/letter/c/da6949/32.png) [@citanionarrei](https://discuss.elastic.co/u/citanionarrei)\
**Post date:** [November 6, 2017, 2:48pm UTC](https://discuss.elastic.co/t/visualization-of-lat-lon-coordinates-on-kibanas-graph/106177/10 "2017-11-06T14:48:17Z")

</div>

how can i create it?  
i'm using this on logstash

```
mutate {
 add_field => { "[geopoint][latitude]" => "%{[venue][lat]}" }
 add_field => { "[geopoint][longitude]" => "%{[venue][lon]}" }
 remove_field =>["meetup","message","group","event"]	}	   
			   
mutate {
 convert => {
   "[geopoint][latitude]" => "float"
   "[geopoint][longitude]" => "float"
}

```

i obtain in my json this:

![Screenshot (54)](https://us1.discourse-cdn.com/elastic/original/3X/a/6/a6b1b8402458116952939a9e4d582a93101b28f7.png)

and on kibana

![Screenshot (55)](https://us1.discourse-cdn.com/elastic/original/3X/9/e/9e9367031e779787ed682ff8a2a45eeda3474131.png)

mapping that i'm using

```
{
 "template":"meetup-*",
"settings":{
"number_of_shards": 1
},

 "mappings": {
  "json_meetup": {
    "properties": {
	  "geopoint": {
	  "type": "geo_point",}
        
}}}}

```

maybe is it broken?

---

<div class="post-metadata">

**Author:** ![mikelD](https://avatars.discourse-cdn.com/v4/letter/m/d26b3c/32.png) [@mikelD](https://discuss.elastic.co/u/mikelD)\
**Post date:** [November 6, 2017, 3:05pm UTC](https://discuss.elastic.co/t/visualization-of-lat-lon-coordinates-on-kibanas-graph/106177/11 "2017-11-06T15:05:31Z")

</div>

In the template after "type": "geo\_point" you have a , that is a syntax mistake and shouldn't allow you to load the template, try using the command `GET _templating/[name of the template]` to check that it has been loaded correctly , the name you have used with the command PUT.

---

<div class="post-metadata">

**Author:** ![citanionarrei](https://avatars.discourse-cdn.com/v4/letter/c/da6949/32.png) [@citanionarrei](https://discuss.elastic.co/u/citanionarrei)\
**Post date:** [November 6, 2017, 5:13pm UTC](https://discuss.elastic.co/t/visualization-of-lat-lon-coordinates-on-kibanas-graph/106177/12 "2017-11-06T17:13:35Z")

</div>

i have problem 😥

my json is:

```
{
 "venue": {
  "venue_name": "Facebook",
  "lon": -6.23911,
  "lat": 53.343712,
  "venue_id": 23843832
},
"visibility": "public",
"rsvp_id": 1697704652,
"geopoint": {
  "latitude": 53.343712,
  "longitude": -6.23911
},
"type": ["log","json_meetup"],
"mtime": 1509981678435,
"path": "C:\\Users\\Lock\\Desktop\\meetup.json",
"@timestamp": "2017-11-06T16:49:04.147Z",
"response": "yes",
"@version": "1",
"host": "Lock",
"guests": 0,
"member": {
  "member_id": 202905630,
  "member_name": "Brian Quinlan"
    }
  },
 "fields": {
"@timestamp": [1509986944147]
}
}

```

my configuration file of logstash is this

```
input 
{file 
{
path => ["C:\Users\Lock\Desktop\meetup.json"]  
start_position => "beginning"
sincedb_path => "/dev/null"
exclude => "*.gz"
type => "log"
}
}

filter{
grok {
match => ["message", "(?<meetup>{\".*\".*})"]
break_on_match => true
add_field => { "type" => "json_meetup"} 	
}
if "_grokparsefailure" in [tags]{drop {}}	
else {
json {source => "meetup"}}	 
mutate {
 add_field => { "[geopoint][latitude]" => "%{[venue][lat]}" }
 add_field => { "[geopoint][longitude]" => "%{[venue][lon]}" }
 remove_field =>["meetup","message","group","event"]	}	   				   
mutate {
 convert => {
   "[geopoint][latitude]" => "float"
   "[geopoint][longitude]" => "float"
}}}			   		   			   
		
output
{ elasticsearch {
index => "meetup-%{+YYYY.MM.dd}"
document_type => "json_meetup"

manage_template => true
template => "C:\Users\Lock\Desktop\template3.json"
template_overwrite => true
codec => json
}

stdout { codec => rubydebug }}

```

and my template that i load on elastic with logstash is this

```
{
  "template":"meetup-*",
"settings":{
"number_of_shards": 1
},

"mappings": {
  "json_meetup": {
    "properties": {
	"geopoint": {
		"properties": {
            "latitude": {
            "type": "geo_point"
          },
          "longitude": {
            "type": "geo_point"
          }
         }
       }
     }
   }
 }
 }

```

in this way mapping dont work

---

<div class="post-metadata">

**Author:** ![mikelD](https://avatars.discourse-cdn.com/v4/letter/m/d26b3c/32.png) [@mikelD](https://discuss.elastic.co/u/mikelD)\
**Post date:** [November 7, 2017, 8:54am UTC](https://discuss.elastic.co/t/visualization-of-lat-lon-coordinates-on-kibanas-graph/106177/13 "2017-11-07T08:54:52Z")

</div>

Your template is wrong,it should be like this:

> {  
> "template":"meetup-\*",  
> "settings":{  
> "number\_of\_shards": 1  
> },
> 
> "mappings": {  
> "json\_meetup": {  
> "properties": {  
> "geopoint": {  
> "type": "geo\_point"  
> }  
> }  
> }  
> }  
> }

You should check the } there seems to be to much, check with a editor like atom to see if they sintaxis of the JSON is correct.

You don't have to define the latitude and longitude as geo\_point but the field that contains them.

---

<div class="post-metadata">

**Author:** ![citanionarrei](https://avatars.discourse-cdn.com/v4/letter/c/da6949/32.png) [@citanionarrei](https://discuss.elastic.co/u/citanionarrei)\
**Post date:** [November 7, 2017, 8:56am UTC](https://discuss.elastic.co/t/visualization-of-lat-lon-coordinates-on-kibanas-graph/106177/14 "2017-11-07T08:56:45Z")

</div>

i have also tried with it but doesn't work 🤨

---

<div class="post-metadata">

**Author:** ![citanionarrei](https://avatars.discourse-cdn.com/v4/letter/c/da6949/32.png) [@citanionarrei](https://discuss.elastic.co/u/citanionarrei)\
**Post date:** [November 7, 2017, 2:42pm UTC](https://discuss.elastic.co/t/visualization-of-lat-lon-coordinates-on-kibanas-graph/106177/15 "2017-11-07T14:42:09Z")

</div>

I solved the problem thanks to you for the help..

im using this template

```
 {
    "template":"meetup-*",
"settings":{
"number_of_shards": 1
			},

"mappings": {
	"log": {
		"properties": {
			"location": {
				"type": "geo_point"
				}
			}
		}
	}
}

```

and this on logstash

```
mutate {

	 add_field => { "[location]" => "%{[venue][lat]},%{[venue][lon]}"}
   }

 kv {
	source => "location"
	target => "location"
	field_split => ","}  
   
 mutate {
	convert => { "[location][lat]" => "float" }
	convert => { "[location][lon]" => "float" }

```

I would have another curiosity: can I map for any field?  
if I wanted to do it for example i want change the type of "guest" from "number "to "string" can i make in this way?

```
 {
    "template":"meetup-*",
"settings":{
"number_of_shards": 1
			},

"mappings": {
	"log": {
		"properties": {
			"location": {
				"type": "geo_point"
				}

             "guests":{
				"type": "text"
				}

			}
		}
	}
}

```

becouse so i have problems and template in this way dont work!

---

<div class="post-metadata">

**Author:** ![mikelD](https://avatars.discourse-cdn.com/v4/letter/m/d26b3c/32.png) [@mikelD](https://discuss.elastic.co/u/mikelD)\
**Post date:** [November 7, 2017, 2:57pm UTC](https://discuss.elastic.co/t/visualization-of-lat-lon-coordinates-on-kibanas-graph/106177/16 "2017-11-07T14:57:45Z")

</div>

I'm not sure if it would work, but even if it worked you would have a problem because the string would be in ASCII and if it converts the number directly as ASCII code it will print other characters. For example: the character '1' in ASCII has the decimal number 49. And the decimal number 1, and many others are special characters like backspace and the like.

You will have to parse the field and convert it to text before sending it.

---

<div class="post-metadata">

**Author:** ![citanionarrei](https://avatars.discourse-cdn.com/v4/letter/c/da6949/32.png) [@citanionarrei](https://discuss.elastic.co/u/citanionarrei)\
**Post date:** [November 7, 2017, 3:00pm UTC](https://discuss.elastic.co/t/visualization-of-lat-lon-coordinates-on-kibanas-graph/106177/17 "2017-11-07T15:00:13Z")

</div>

yes it was only an example..  
i speaked for the template and if the structure for the mapping is correct.  
it is the way to map all fields?

---

<div class="post-metadata">

**Author:** ![mikelD](https://avatars.discourse-cdn.com/v4/letter/m/d26b3c/32.png) [@mikelD](https://discuss.elastic.co/u/mikelD)\
**Post date:** [November 8, 2017, 10:49am UTC](https://discuss.elastic.co/t/visualization-of-lat-lon-coordinates-on-kibanas-graph/106177/18 "2017-11-08T10:49:22Z")

</div>

Yes, you can map for nay kind of field, automatically elasticsearch detects the type of data and maps it to the index, but some special types like geo\_point and ip must be mapped manually. In my experience, I recommend mapping all the fields in the template to avoid problems if it maps a field incorrectly.

About the structure you have to separate the diferent fiels with a ',', like this:

> "batterylevel": {  
> "type": "long"  
> },  
> "longitude": {  
> "type": "float"  
> }

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 6, 2017, 10:50am UTC](https://discuss.elastic.co/t/visualization-of-lat-lon-coordinates-on-kibanas-graph/106177/19 "2017-12-06T10:50:18Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.