# How do you make geo\_points in Elasticsearch 6.x?

**URL:** <https://discuss.elastic.co/t/how-do-you-make-geo-points-in-elasticsearch-6-x/112729>\
**Category:** Elasticsearch\
**Created:** [December 21, 2017, 12:28am UTC](https://discuss.elastic.co/t/how-do-you-make-geo-points-in-elasticsearch-6-x/112729 "2017-12-21T00:28:21Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![tehowe](https://avatars.discourse-cdn.com/v4/letter/t/a3d4f5/32.png) [@tehowe](https://discuss.elastic.co/u/tehowe)\
**Post date:** [December 21, 2017, 12:28am UTC](https://discuss.elastic.co/t/how-do-you-make-geo-points-in-elasticsearch-6-x/112729/1 "2017-12-21T00:28:21Z")

</div>

Hello, so I think I so far have a pretty good basic foundation for my central logging project set up.

I have a number of hosts running filebeat, shipping syslogs, apache2 logs, and the apt history log to one of two identical logstash servers, which then filter the data and forward it on to a three-node Elasticsearch cluster. (My configurations are below.)

I'm a bit baffled as to why the logstash geoip plugin doesn't create the geo\_point type automatically with the rest of the geoip information however.

In Kibana, attempting to create a map results in the dread message

> No Compatible Fields: The "filebeat-\*" index pattern does not contain any of the following field types: geo\_point

From what I've read, it seems that the way to resolve this problem on the 6.x ELK stack is to make a new index template for filebeat. But when I

```
curl -XGET 'my_elasticsearch_node:9200/_template/*?pretty'

```

The only template I see by default is for Kibana. Which is confusing because everything I've read so far (mostly for much older versions of ELK, granted) implies it should have an existing filebeat template in order to index filebeat stuff, especially given that my logstash servers are set to

```
manage_template => false

```

in its output clause. So I've read the general index template doc at [https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-templates.html#indices-templates-exists](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-templates.html#indices-templates-exists) and the Digital Ocean guide at [https://www.digitalocean.com/community/tutorials/how-to-map-user-location-with-geoip-and-elk-elasticsearch-logstash-and-kibana](https://www.digitalocean.com/community/tutorials/how-to-map-user-location-with-geoip-and-elk-elasticsearch-logstash-and-kibana) but I don't really understand how to apply them in this case.

How do you conceptualize how these moving parts work together? Could someone please explain that and walk me through how to get geo\_points working?

**filebeat config**  
/etc/filebeat/filebeat.yml

> filebeat.prospectors:  
> - input\_type: log  
> paths:
> 
> - /var/log/\*.log  
> fields:  
> type: syslog  
> - input\_type: log  
> paths:
> - /var/log/apache2/\*.log  
> fields:  
> type: apache2  
> - input\_type: log  
> paths:
> - /var/log/apt/history.log  
> fields:  
> type: apt  
> multiline.pattern: Start-Date  
> multiline.negate: true  
> multiline.match: after  
> multiline.flush\_pattern: End-Date
> 
> output.logstash:  
> hosts: ["logstash1\_ip:5044", "logstash2\_ip:5044"]  
> ssl.enabled: true  
> ssl.supported\_protocols: [TLSv1.2]  
> ssl.certificate\_authorities: ["/etc/filebeat/ca\_keys/ca-chaincert.pem"]  
> ssl.certificate: "/etc/filebeat/ca\_keys/hostcert.pem"  
> ssl.key: "/etc/filebeat/ca\_keys/hostkey.pem"

**logstash config**  
/etc/logstash/conf.d/logstash.conf

> input {  
> beats {  
> port =\> 5044  
> ssl =\> true  
> ssl\_certificate\_authorities =\> ["/etc/logstash/ca\_keys/ca-chaincert.pem"]  
> ssl\_certificate =\> "/etc/logstash/ca\_keys/logstash1cert.pem"  
> ssl\_key =\> "/etc/logstash/ca\_keys/logstash1key.pem"  
> ssl\_verify\_mode =\> "force\_peer"  
> client\_inactivity\_timeout =\> 180  
> }  
> }
> 
> filter {  
> if [fields][type] == "syslog" {  
> grok {  
> match =\> { message =\> "%{SYSLOGTIMESTAMP:timestamp} %{SYSLOGHOST:host} %{DATA:syslog\_program}(?:[%{POSINT:syslog\_pid}])?: %{GREEDYDATA:msg}"  
> }  
> }  
> date {  
> match =\> ["timestamp" , "MMM dd HH:mm:ss"]  
> }  
> }  
> else if [fields][type] == "apache2" {  
> grok {  
> match =\> { message =\> "%{COMBINEDAPACHELOG}"  
> }  
> }  
> geoip {  
> source =\> "clientip"  
> }  
> date {  
> match =\> ["timestamp" , "dd/MMM/yyyy:HH:mm:ss Z"]  
> }  
> }  
> else if [fields][type] == "apt" {  
> grok {  
> match =\> { message =\> "Start-Date: %{YEAR:year}-%{MONTHNUM:month}-%{MONTHDAY:day} %{HOUR}:%{MINUTE}:%{SECOND} Commandline: %{GREEDYDATA:commandline} (Upgrade: %{GREEDYDATA:upgrade})?(Install: %{GREEDYDATA:install})? End-Date: %{GREEDYDATA:junk}"  
> }  
> }  
> mutate {  
> add\_field =\> {"timestamp" =\> "%{year}-%{month}-%{day} %{time}"}  
> remove\_field =\> ['junk']  
> }  
> date {  
> match =\> ["timestamp" , "yyyy-MM-dd HH:mm:ss"]  
> }  
> }  
> }
> 
> output {  
> elasticsearch {  
> hosts =\> ["es\_node1\_ip:9200", "es\_node2\_ip:9200", "es\_node3\_ip:9200"]  
> manage\_template =\> false  
> index =\> "%{[@metadata][beat]}-%{+YYYY.MM.dd}"  
> document\_type =\> "%{[@metadata][type]}"  
> }  
> }

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [December 21, 2017, 5:43am UTC](https://discuss.elastic.co/t/how-do-you-make-geo-points-in-elasticsearch-6-x/112729/2 "2017-12-21T05:43:14Z")

</div>

[https://www.elastic.co/blog/geoip-in-the-elastic-stack](https://www.elastic.co/blog/geoip-in-the-elastic-stack) runs through how to get things working, as well as some commonly seen errors.

However it doesn't look like you have a `geoip` filter defined anywhere?

---

<div class="post-metadata">

**Author:** ![tehowe](https://avatars.discourse-cdn.com/v4/letter/t/a3d4f5/32.png) [@tehowe](https://discuss.elastic.co/u/tehowe)\
**Post date:** [December 21, 2017, 7:05pm UTC](https://discuss.elastic.co/t/how-do-you-make-geo-points-in-elasticsearch-6-x/112729/3 "2017-12-21T19:05:13Z")

</div>

(geoip is actually in my logstash filter)

Thanks for your response! your blog post helped a lot 🙂  
This one was also extremely helpful to get a better handle on how mappings work  
[https://www.elastic.co/blog/logstash\_lesson\_elasticsearch\_mapping](https://www.elastic.co/blog/logstash_lesson_elasticsearch_mapping)

Could I suggest that some of the salient excerpts make it into the documentation site where they'd be easier to find?

So the exact steps I took to get this working (for the benefit of anyone else that makes it to this topic) were

1. Copy this file to the root of one of your elasticsearch nodes - click on raw so it's easier to copy/paste  
[https://github.com/logstash-plugins/logstash-output-elasticsearch/blob/master/lib/logstash/outputs/elasticsearch/elasticsearch-template-es6x.json](https://github.com/logstash-plugins/logstash-output-elasticsearch/blob/master/lib/logstash/outputs/elasticsearch/elasticsearch-template-es6x.json)

2. I had to edit the top line which reads

> "template" : "logstash-\*",

to read

> "template" : "filebeat-\*",

1. Now apply this mapping (note that the -H flag is omitted in the blogs and docs I saw but needs to be there or you get a 406 error complaining about the Content-Type)

> curl -XPUT http://elasticsearch\_node1:9200/\_template/filebeat\_template?pretty -H 'Content-Type: application/json' -d @elasticsearch-template-es6x.json

1. Blow away your existing data with

> curl -XDELETE 'elasticsearch\_node1:9200/\_all?pretty'

1. You're done! The new indices flowing into Elasticsearch will include a geo\_point

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [December 22, 2017, 2:45am UTC](https://discuss.elastic.co/t/how-do-you-make-geo-points-in-elasticsearch-6-x/112729/4 "2017-12-22T02:45:50Z")

</div>

> [@tehowe](#):
>
> (geoip is actually in my logstash filter)

Yeah I somehow missed that ☹

> [@tehowe](#):
>
> Could I suggest that some of the salient excerpts make it into the documentation site where they'd be easier to find?

Did you see [Configure Elasticsearch index template loading | Filebeat Reference [8.11] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/configuration-template.html)? I am wondering if it covers this?

> [@tehowe](#):
>
> curl -XDELETE 'elasticsearch\_node1:9200/\_all?pretty'

I wouldn't do that, it removes everything, including any dashboards you may have built. Try to target specific indices or patterns.

---

<div class="post-metadata">

**Author:** ![Haydz](https://avatars.discourse-cdn.com/v4/letter/h/9d8465/32.png) [@Haydz](https://discuss.elastic.co/u/Haydz)\
**Post date:** [December 22, 2017, 4:30pm UTC](https://discuss.elastic.co/t/how-do-you-make-geo-points-in-elasticsearch-6-x/112729/5 "2017-12-22T16:30:26Z")

</div>

> [@warkolm](#):
>
> I wouldn't do that, it removes everything, including any dashboards you may have built. Try to target specific indices or patterns.

Thanks for the heads up! Will keep that in mind.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 19, 2018, 4:30pm UTC](https://discuss.elastic.co/t/how-do-you-make-geo-points-in-elasticsearch-6-x/112729/6 "2018-01-19T16:30:32Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.