# Geoip create everything but the geoip.location

**URL:** <https://discuss.elastic.co/t/geoip-create-everything-but-the-geoip-location/109900>\
**Category:** Logstash\
**Created:** [December 1, 2017, 9:25am UTC](https://discuss.elastic.co/t/geoip-create-everything-but-the-geoip-location/109900 "2017-12-01T09:25:15Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![Adesfire](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adesfire/32/24765_2.png) [@Adesfire](https://discuss.elastic.co/u/Adesfire)\
**Post date:** [December 1, 2017, 9:25am UTC](https://discuss.elastic.co/t/geoip-create-everything-but-the-geoip-location/109900/1 "2017-12-01T09:25:15Z")

</div>

Good morning everyone,  
today I'm trying to implement the geoip filter, using this input file :

```
    filter {
        if [host] =~ /10\.0\.0\.1/ {

            grok {
                    match => ["message", "\A%{SYSLOG5424PRI}%{SYSLOGTIMESTAMP}( )%{WORD}(: )(?:[0-9])?,(?:[0-9]*,)?(?:[0-9]*,)?(?:[0-9]*,)?%{WORD:interface},%{WORD},%{WORD:action},%{WORD:direction},(?:[0-9]*,)(?:[0-9a-z]*,)?(?:[0-9]*,)?(?:[0-9]*,)?(?:[0-9]*,)?(?:[0-9]*,)?%{WORD},(?:[0-9]*,)?%{WORD:transport},(?:[0-9]*,)?%{IP:ipSource},%{IP:ipDestination},%{GREEDYDATA}"]
            }

            geoip {
                    add_tag => ["GeoIP"]
                    source => "ipSource"
            }
    }

```

}

If I got many of the Geoip fields (country name, latitude, longitude, etc.) the only one missing field is the geoip.location as you can see in this screenshot :

 ![fwevent](https://us1.discourse-cdn.com/elastic/original/3X/d/6/d63a3ac66340c41a2220272d81951108ce8abe44.JPG)

Any idea about this problem ?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [December 1, 2017, 9:27am UTC](https://discuss.elastic.co/t/geoip-create-everything-but-the-geoip-location/109900/2 "2017-12-01T09:27:06Z")

</div>

Because it's an RFC1918 address - [https://en.wikipedia.org/wiki/Private\_network](https://en.wikipedia.org/wiki/Private_network)

---

<div class="post-metadata">

**Author:** ![Adesfire](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adesfire/32/24765_2.png) [@Adesfire](https://discuss.elastic.co/u/Adesfire)\
**Post date:** [December 1, 2017, 9:30am UTC](https://discuss.elastic.co/t/geoip-create-everything-but-the-geoip-location/109900/3 "2017-12-01T09:30:22Z")

</div>

I don't think so that 73.112.73.28 is a private address ! Anyway if I got the latitude and longitude of this address, why geoip can't build a geoip.location with it ?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [December 1, 2017, 9:34am UTC](https://discuss.elastic.co/t/geoip-create-everything-but-the-geoip-location/109900/4 "2017-12-01T09:34:45Z")

</div>

It is there, but has two subfields - `geoip.location.lat` and `geoip.location.lon`.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [December 1, 2017, 9:48am UTC](https://discuss.elastic.co/t/geoip-create-everything-but-the-geoip-location/109900/5 "2017-12-01T09:48:06Z")

</div>

Oh, I was looking at the `if [host] =~ /10\.0\.0\.1/`.

Didn't even look at the picture sorry.

---

<div class="post-metadata">

**Author:** ![Adesfire](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adesfire/32/24765_2.png) [@Adesfire](https://discuss.elastic.co/u/Adesfire)\
**Post date:** [December 1, 2017, 9:51am UTC](https://discuss.elastic.co/t/geoip-create-everything-but-the-geoip-location/109900/6 "2017-12-01T09:51:46Z")

</div>

No because of a few days ago, on the same server, this plugin generated the geoip.location, first with a wrong type but next like a charm.  
Why is the hell Logstash doing the things differently now? I didn't change the line of my grok, I only deleted every indexes to start from a fresh one.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [December 1, 2017, 10:01am UTC](https://discuss.elastic.co/t/geoip-create-everything-but-the-geoip-location/109900/7 "2017-12-01T10:01:41Z")

</div>

Are you having Logstash manage the index templates? Have you got an index template that apply the correct mappings for this field for your `pfsense` index (Logstash by default assumes indices matching `logstash-*` in the default index template)?

---

<div class="post-metadata">

**Author:** ![Adesfire](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adesfire/32/24765_2.png) [@Adesfire](https://discuss.elastic.co/u/Adesfire)\
**Post date:** [December 1, 2017, 10:17am UTC](https://discuss.elastic.co/t/geoip-create-everything-but-the-geoip-location/109900/8 "2017-12-01T10:17:41Z")

</div>

I think you have found something! Before today, I had only one index (logstash-\*) but now I used IP to conditionally split into different indexes. So, for my Pfsense ip server, I got this output :

```
 "location": {
                "properties": {
                  "lat": {
                    "type": "float"
                  },
                  "lon": {
                    "type": "float"
                  }
                }
              },
```

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [December 1, 2017, 10:29am UTC](https://discuss.elastic.co/t/geoip-create-everything-but-the-geoip-location/109900/9 "2017-12-01T10:29:10Z")

</div>

That is indeed not correct. You can copy the default Logstash template, change the index pattern it matches and then upload it using a different name. You will need to reindex the data though.

---

<div class="post-metadata">

**Author:** ![Adesfire](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adesfire/32/24765_2.png) [@Adesfire](https://discuss.elastic.co/u/Adesfire)\
**Post date:** [December 1, 2017, 11:06am UTC](https://discuss.elastic.co/t/geoip-create-everything-but-the-geoip-location/109900/10 "2017-12-01T11:06:45Z")

</div>

Ok that sounds good, but I'm unable to make a copy of the logstash template, I'm a newbie 😖  
I read many posts, but that seems to have changed (that never work)

Could you help me to do that operation, with the latest version of ELK ?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [December 1, 2017, 11:12am UTC](https://discuss.elastic.co/t/geoip-create-everything-but-the-geoip-location/109900/11 "2017-12-01T11:12:48Z")

</div>

The template can be found [here](https://github.com/logstash-plugins/logstash-output-elasticsearch/blob/master/lib/logstash/outputs/elasticsearch/elasticsearch-template-es6x.json). Download this and change `"template" : "logstash-*",` to match your index name. Then upload this [index template](https://www.elastic.co/guide/en/elasticsearch/reference/6.0/indices-templates.html) with a suitable name. It should then apply to all new indices created that match the specified pattern.

---

<div class="post-metadata">

**Author:** ![Adesfire](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adesfire/32/24765_2.png) [@Adesfire](https://discuss.elastic.co/u/Adesfire)\
**Post date:** [December 1, 2017, 2:36pm UTC](https://discuss.elastic.co/t/geoip-create-everything-but-the-geoip-location/109900/12 "2017-12-01T14:36:22Z")

</div>

Ok, thank you very much, with your help I have been able to do the job!  
As a beginner with ALK, i found that's very difficult to create and update indexes and data type. Copy/edit/paste using curl commands, then delete indexes, create a new one, etc. It's not really natural and easy, especially when the syntax evolved so forums threads can't help you.

Have you think about simplifying this process ?

Anyway, this project is awesome and you too !

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [December 5, 2017, 6:41am UTC](https://discuss.elastic.co/t/geoip-create-everything-but-the-geoip-location/109900/13 "2017-12-05T06:41:41Z")

</div>

> ```
> if [host] =~ /10\.0\.0\.1/ {
> 
> ```

Using a plain string comparison will be faster and won't incorrectly match addresses like 110.0.0.11.

---

<div class="post-metadata">

**Author:** ![Adesfire](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adesfire/32/24765_2.png) [@Adesfire](https://discuss.elastic.co/u/Adesfire)\
**Post date:** [December 5, 2017, 7:23am UTC](https://discuss.elastic.co/t/geoip-create-everything-but-the-geoip-location/109900/14 "2017-12-05T07:23:55Z")

</div>

Thanks for this tip!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 2, 2018, 7:24am UTC](https://discuss.elastic.co/t/geoip-create-everything-but-the-geoip-location/109900/15 "2018-01-02T07:24:19Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.