# Filebeat to logstash geo\_point missing

**URL:** <https://discuss.elastic.co/t/filebeat-to-logstash-geo-point-missing/153080>\
**Category:** Logstash\
**Created:** [October 18, 2018, 10:31pm UTC](https://discuss.elastic.co/t/filebeat-to-logstash-geo-point-missing/153080 "2018-10-18T22:31:52Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![rajsolanki](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rajsolanki/32/86859_2.png) [@rajsolanki](https://discuss.elastic.co/u/rajsolanki)\
**Post date:** [October 18, 2018, 10:31pm UTC](https://discuss.elastic.co/t/filebeat-to-logstash-geo-point-missing/153080/1 "2018-10-18T22:31:52Z")

</div>

Ok Please pardon me for asking same question regarding geo\_point. I have done tons of reading in this forum and have tried everything i can think of after reading but i just cant find a solution to my issue.

So here it is.

I am using file beat to ingest IBM HTTP server to Logstash. Logstash config has following filter

filter {  
grok {  
match =\> { "message" =\> '%{IPORHOST:clientip} %{USER:ident} %{USER:auth} [%{HTTPDATE:timestamp}] %{WORD:verb} %{URIPATHPARAM:request} HTTP/%{NUMBER:httpversion} %{NUMBER:response} (?:%{NUMBER:bytes}|-) %{NUMBER:responsetime} "%{DATA:referrer}" "%{DATA:agent}"' }  
remove\_field =\> ["message"]  
}

geoip {  
source =\> "clientip"  
target =\> "geoip"  
add\_tag =\> ["webserver-geoip"]  
}

if "\_grokparsefailure" in [tags] {  
drop { }  
}  
date {  
match =\> ["timestamp", "dd/MMM/yyyy:HH:mm:ss Z"]  
remove\_field =\> ["timestamp"]  
}  
mutate {  
convert =\> {  
"bytes" =\> "integer"  
"response" =\> "integer"  
"responsetime" =\> "float"  
}  
}  
}

output {  
elasticsearch {  
hosts =\> ["[http://xx.xx.xx.xx](http://xx.xx.xx.xx):xxxxx" ]  
template =\> "/mytemplate/template.json"  
template\_overwrite =\> true  
index =\> "mywebservers-%{+YYYY.MM.dd}"  
}  
}

Here is my template.json

{  
"template": "logstash-_",  
"settings": {"index.refresh\_interval": "-1"},  
"mappings": {  
"default": {  
"\_all": {"enabled": false},  
"date\_detection": false,  
"dynamic\_templates": [  
{"string\_fields": {  
"match": "_",  
"match\_mapping\_type": "string",  
"mapping": {"type": "keyword"}  
}}  
],  
"properties": {  
"@timestamp": {"type": "date", "format": "dateOptionalTime"},  
"agent": {"type": "text", "fields": {"raw": {"type": "keyword"}}},  
"referrer": {"type": "text", "fields": {"raw": {"type": "keyword"}}},  
"request": {"type": "text", "fields": {"raw": {"type": "keyword"}}},  
"host": {"type": "keyword"},  
"httpversion": {"type": "keyword"},  
"user": {"type": "keyword"},  
"operation": {"type": "keyword"},  
"bytes": {"type": "long"},  
"response": {"type": "short"},  
"responsetime":{"type":"long" },  
"clientip": {"type": "ip"},  
"geoip" : {  
"dynamic": true,  
"properties" : {  
"ip": { "type": "ip" },  
"location" : { "type" : "geo\_point" },  
"latitude" : { "type" : "float" },  
"longitude" : { "type" : "float" }  
}  
}  
}  
}  
}  
}

I am able to see all documents which has geo ip related details populated. Enitre json record has all geo related values populated. But there is no geo\_point. My template.json is taking care of it but in visualization when i try to create coordinated map and select geo hash it gives me this error.

**No Compatible Fields:** The "mywebservers\*" index pattern does not contain any of the following field types: geo\_point

have i missed something ? is it very simple solution where in template.json "template": "logstash-_", should be mywebservers_ ?

thanks

Raj

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [October 18, 2018, 10:33pm UTC](https://discuss.elastic.co/t/filebeat-to-logstash-geo-point-missing/153080/2 "2018-10-18T22:33:34Z")

</div>

What does the mapping for the index look like?

---

<div class="post-metadata">

**Author:** ![rajsolanki](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rajsolanki/32/86859_2.png) [@rajsolanki](https://discuss.elastic.co/u/rajsolanki)\
**Post date:** [October 19, 2018, 12:38am UTC](https://discuss.elastic.co/t/filebeat-to-logstash-geo-point-missing/153080/3 "2018-10-19T00:38:22Z")

</div>

"other-webservers-ppd-2018.10.17": {  
"mappings": {  
"doc": {  
"properties": {  
"@timestamp": {  
"type": "date"  
},  
"@version": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
},  
"agent": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
},  
"auth": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
},  
"beat": {  
"properties": {  
"hostname": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
},  
"name": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
},  
"version": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
}  
}  
},  
"bytes": {  
"type": "long"  
},  
"clientip": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
},  
"geoip": {  
"properties": {  
"city\_name": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
},  
"continent\_code": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
},  
"country\_code2": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
},  
"country\_code3": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
},  
"country\_name": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
},  
"dma\_code": {  
"type": "long"  
},  
"ip": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
},  
"latitude": {  
"type": "float"  
},  
"location": {  
"properties": {  
"lat": {  
"type": "float"  
},  
"lon": {  
"type": "float"  
}  
}  
},  
"longitude": {  
"type": "float"  
},  
"postal\_code": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
},  
"region\_code": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
},  
"region\_name": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
},  
"timezone": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
}  
}  
},  
"host": {  
"properties": {  
"name": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
}  
}  
},  
"httpversion": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
},  
"ident": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
},  
"input": {  
"properties": {  
"type": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
}  
}  
},  
"offset": {  
"type": "long"  
},  
"prospector": {  
"properties": {  
"type": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
}  
}  
},  
"referrer": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
},

---

<div class="post-metadata">

**Author:** ![rajsolanki](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rajsolanki/32/86859_2.png) [@rajsolanki](https://discuss.elastic.co/u/rajsolanki)\
**Post date:** [October 19, 2018, 12:38am UTC](https://discuss.elastic.co/t/filebeat-to-logstash-geo-point-missing/153080/4 "2018-10-19T00:38:50Z")

</div>

"request": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
},  
"response": {  
"type": "long"  
},  
"responsetime": {  
"type": "float"  
},  
"source": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
},  
"tags": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
},  
"verb": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
}  
}  
}  
}  
}

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [October 19, 2018, 2:50am UTC](https://discuss.elastic.co/t/filebeat-to-logstash-geo-point-missing/153080/5 "2018-10-19T02:50:39Z")

</div>

It'd be easier to read if you format things using the `</>` button, or put markdown code backticks around the code.

---

<div class="post-metadata">

**Author:** ![rajsolanki](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rajsolanki/32/86859_2.png) [@rajsolanki](https://discuss.elastic.co/u/rajsolanki)\
**Post date:** [October 19, 2018, 2:21pm UTC](https://discuss.elastic.co/t/filebeat-to-logstash-geo-point-missing/153080/6 "2018-10-19T14:21:36Z")

</div>

apologies for not using \</\>. I think i have narrowed it down. If i dont use template at all logstash uses default one. so when i start logstash with no template i get this output.

Attempting to install template {:manage\_template=\>{"template"=\>"logstash-_", "version"=\>60001, "settings"=\>{"index.refresh\_interval"=\>"5s"}, "mappings"=\>{"default"=\>{"dynamic\_templates"=\>[{"message\_field"=\>{"path\_match"=\>"message", "match\_mapping\_type"=\>"string", "mapping"=\>{"type"=\>"text", "norms"=\>false}}}, {"string\_fields"=\>{"match"=\>"_", "match\_mapping\_type"=\>"string", "mapping"=\>{"type"=\>"text", "norms"=\>false, "fields"=\>{"keyword"=\>{"type"=\>"keyword", "ignore\_above"=\>256}}}}}], "properties"=\>{"@timestamp"=\>{"type"=\>"date"}, "@version"=\>{"type"=\>"keyword"}, "geoip"=\>{"dynamic"=\>true, "properties"=\>{"ip"=\>{"type"=\>"ip"}, "location"=\>{"type"=\>"geo\_point"}, "latitude"=\>{"type"=\>"half\_float"}, "longitude"=\>{"type"=\>"half\_float"}}}}}}}}

where as if i use my template i see this output.

:manage\_template=\>{"template"=\>"logstash-_", "settings"=\>{"index.refresh\_interval"=\>"-1"}, "mappings"=\>{"default"=\>{"\_all"=\>{"enabled"=\>false}, "date\_detection"=\>false, "dynamic\_templates"=\>[{"string\_fields"=\>{"match"=\>"_", "match\_mapping\_type"=\>"string", "mapping"=\>{"type"=\>"keyword"}}}], "properties"=\>{"@timestamp"=\>{"type"=\>"date", "format"=\>"dateOptionalTime"}, "agent"=\>{"type"=\>"text", "fields"=\>{"raw"=\>{"type"=\>"keyword"}}}, "referrer"=\>{"type"=\>"text", "fields"=\>{"raw"=\>{"type"=\>"keyword"}}}, "request"=\>{"type"=\>"text", "fields"=\>{"raw"=\>{"type"=\>"keyword"}}}, "host"=\>{"type"=\>"keyword"}, "httpversion"=\>{"type"=\>"keyword"}, "dealer"=\>{"type"=\>"keyword"}, "operation"=\>{"type"=\>"keyword"}, "bytes"=\>{"type"=\>"long"}, "response"=\>{"type"=\>"short"}, "responsetime"=\>{"type"=\>"long"}, "clientip"=\>{"type"=\>"ip"}, "geoip"=\>{"dynamic"=\>true, "properties"=\>{"ip"=\>{"type"=\>"ip"}, "location"=\>{"type"=\>"geo\_point"}, "latitude"=\>{"type"=\>"float"}, "longitude"=\>{"type"=\>"float"}}}}}}}}

I have verified my format its valid json. only diff i see is longitude and latitude are half float in default where as float in mine. can that be it ?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 16, 2018, 2:22pm UTC](https://discuss.elastic.co/t/filebeat-to-logstash-geo-point-missing/153080/7 "2018-11-16T14:22:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.