# Can we create two GeoIP Filters in one logstash config file?

**URL:** <https://discuss.elastic.co/t/can-we-create-two-geoip-filters-in-one-logstash-config-file/42849>\
**Category:** Logstash\
**Created:** [February 26, 2016, 12:55pm UTC](https://discuss.elastic.co/t/can-we-create-two-geoip-filters-in-one-logstash-config-file/42849 "2016-02-26T12:55:09Z")\
**Posts on this page:** 18\
**Page:** 1

<div class="post-metadata">

**Author:** ![vikas\_gopal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vikas_gopal/32/47661_2.png) [@vikas\_gopal](https://discuss.elastic.co/u/vikas_gopal)\
**Post date:** [February 26, 2016, 12:55pm UTC](https://discuss.elastic.co/t/can-we-create-two-geoip-filters-in-one-logstash-config-file/42849/1 "2016-02-26T12:55:09Z")

</div>

Hi Experts,

My requirement is to create 2 maps , one is for Source IP and other is for Destination IP.

For Source IP what I have done is I used GeoIP filter as below  
geoip { source =\> "src"}

Now I am trying the same for Destination geoip { source =\> "dst"}, but in the map visualization I can only see geoip.location, now confusion is how I can select for src or dst.

Thanks  
VG

---

<div class="post-metadata">

**Author:** ![vikas\_gopal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vikas_gopal/32/47661_2.png) [@vikas\_gopal](https://discuss.elastic.co/u/vikas_gopal)\
**Post date:** [February 26, 2016, 1:05pm UTC](https://discuss.elastic.co/t/can-we-create-two-geoip-filters-in-one-logstash-config-file/42849/2 "2016-02-26T13:05:35Z")

</div>

So I have done this but still in Kibana I do not see two fileds.

filter {  
geoip {  
source =\> "src"  
target =\> "src\_geoip"  
}  
geoip {  
source =\> "dst"  
target =\> "dst\_geoip"  
}  
}

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 26, 2016, 2:33pm UTC](https://discuss.elastic.co/t/can-we-create-two-geoip-filters-in-one-logstash-config-file/42849/3 "2016-02-26T14:33:12Z")

</div>

That's because those fields aren't mapped as geo\_point. You need to adjust the Logstash index template. See the related options in the documentation of the elasticsearch output.

---

<div class="post-metadata">

**Author:** ![vikas\_gopal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vikas_gopal/32/47661_2.png) [@vikas\_gopal](https://discuss.elastic.co/u/vikas_gopal)\
**Post date:** [February 26, 2016, 2:38pm UTC](https://discuss.elastic.co/t/can-we-create-two-geoip-filters-in-one-logstash-config-file/42849/4 "2016-02-26T14:38:53Z")

</div>

@magnusbaeck,

Thanks for the reply , so I have following index template.  
"geoip" : {  
"type" : "object",  
"dynamic": true,  
"path": "full",  
"properties" : {  
"location" : { "type" : "geo\_point" }  
}  
}

Do I need to make any changes to it ?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 26, 2016, 2:41pm UTC](https://discuss.elastic.co/t/can-we-create-two-geoip-filters-in-one-logstash-config-file/42849/5 "2016-02-26T14:41:55Z")

</div>

Yes. That mapping is for fields named `geoip`. Your fields are named `src_geoip` and `dst_geoip`.

---

<div class="post-metadata">

**Author:** ![vikas\_gopal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vikas_gopal/32/47661_2.png) [@vikas\_gopal](https://discuss.elastic.co/u/vikas_gopal)\
**Post date:** [February 26, 2016, 3:09pm UTC](https://discuss.elastic.co/t/can-we-create-two-geoip-filters-in-one-logstash-config-file/42849/6 "2016-02-26T15:09:19Z")

</div>

@magnusbaeck Ah!!! Now I get what you are saying , it works for me . Thanks alot, you are always helpful .

---

<div class="post-metadata">

**Author:** ![Cody\_Betsworth](https://avatars.discourse-cdn.com/v4/letter/c/57b2e6/32.png) [@Cody\_Betsworth](https://discuss.elastic.co/u/Cody_Betsworth)\
**Post date:** [July 14, 2016, 9:20pm UTC](https://discuss.elastic.co/t/can-we-create-two-geoip-filters-in-one-logstash-config-file/42849/7 "2016-07-14T21:20:55Z")

</div>

Good Afternoon,

Forgive me for bringing an old thread back to life but I am still struggling with how to get this to work for both dst and src ip addresses for a geoip lookup. I have

filter {  
geoip {  
source =\> "src"  
target =\> "src\_geoip"  
}  
geoip {  
source =\> "dst"  
target =\> "dst\_geoip"  
}  
}

and the corresponding fields come into the index inside of elasticsearch just fine. The src\_geoip.location and dst\_geoip.location fields look to be in the correct format for geojson. Can you assist me in what commands I need to enter to change the type of these two fields in my mapping to geo points and not numbers as they stand today.  
We are currently just using the default index that is generated from the elasticsearch output from logstash. My geopoint field looks the same as above.

Please get back to me at your earliest convenience. Thank you.

Cody Betsworth

---

<div class="post-metadata">

**Author:** ![Cody\_Betsworth](https://avatars.discourse-cdn.com/v4/letter/c/57b2e6/32.png) [@Cody\_Betsworth](https://discuss.elastic.co/u/Cody_Betsworth)\
**Post date:** [July 14, 2016, 9:22pm UTC](https://discuss.elastic.co/t/can-we-create-two-geoip-filters-in-one-logstash-config-file/42849/8 "2016-07-14T21:22:08Z")

</div>

If you need me to provide any additional information just ask and I can provide whatever you need to assist. Been struggling with this one for awhile. Thank you in advance to anyone willing to help.

Cody

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [July 15, 2016, 5:39am UTC](https://discuss.elastic.co/t/can-we-create-two-geoip-filters-in-one-logstash-config-file/42849/9 "2016-07-15T05:39:09Z")

</div>

You need to update the index template used for your indexes to map the `src_geoip` and `dst_geoip` fields as geo\_point. By default Logstash's elasticsearch output uploads the index template to use and there are a few configuration options that control the exact behavior. Make a copy of the current index template and make the necessary adjustments there.

For further help please ask a more specific question. At least I don't have time to write a full step-by-step for this, but I'm sure it's been before by other people (here, in the documentation, in blog posts and/or on StackOverflow).

---

<div class="post-metadata">

**Author:** ![vikas\_gopal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vikas_gopal/32/47661_2.png) [@vikas\_gopal](https://discuss.elastic.co/u/vikas_gopal)\
**Post date:** [July 15, 2016, 7:57am UTC](https://discuss.elastic.co/t/can-we-create-two-geoip-filters-in-one-logstash-config-file/42849/11 "2016-07-15T07:57:53Z")

</div>

Hi Cody,

This is what you need to do . Assuming you are using ES1.7.X, because things are slight different if you are on ES2.X

step 1 --\> In LS you need to update target for src\_geoip and dst\_geoip as geoip , something like (be sure src and dst fields should be IP type)

geoip { source =\> "src" target =\> "srcgeoip" }  
geoip {source =\> "dst" target =\> "dstgeoip" }

Step 2) In ES template or using API you need to map fields to Geo\_point

"srcgeoip" : {"type" : "object","dynamic": true,"path": "full","properties" : {"location" : { "type" : "geo\_point" }}},  
"dstgeoip" : {"type" : "object","dynamic": true,"path": "full","properties" : {"location" : { "type" : "geo\_point" }}}

Step 3) now parse your data and you will see something like this in Kibana4.1.1  
 ![](https://us1.discourse-cdn.com/elastic/original/2X/a/a25654259b84c7438bfd191703df79f1893a5355.png)

Let me know if you have more queries or concern on this

Thanks  
VG

---

<div class="post-metadata">

**Author:** ![Cody\_Betsworth](https://avatars.discourse-cdn.com/v4/letter/c/57b2e6/32.png) [@Cody\_Betsworth](https://discuss.elastic.co/u/Cody_Betsworth)\
**Post date:** [July 15, 2016, 2:32pm UTC](https://discuss.elastic.co/t/can-we-create-two-geoip-filters-in-one-logstash-config-file/42849/12 "2016-07-15T14:32:50Z")

</div>

**I am currently using elasticsearch 2.3.**

{  
"name" : "elk02",  
"cluster\_name" : "ArtOfSteal2",  
"version" : {  
"number" : "2.3.3",  
"build\_hash" : "218bdf10790eef486ff2c41a3df5cfa32dadcfde",  
"build\_timestamp" : "2016-05-17T15:40:04Z",  
"build\_snapshot" : false,  
"lucene\_version" : "5.5.0"  
},  
"tagline" : "You Know, for Search"  
}

**Here is my elasticsearch-fortinet.json template I am trying to use. I tried to just simply add your recommendations below the geoip field below but it still does not seem to be working. Thank you again for all your help. You mentioned 2.x elasticsearch is a little different on how I need to implement this request. Get back to me at your earliest convenience.**

{  
"template" : "fortinet\*",  
"settings" : {  
"index.refresh\_interval" : "5s"  
},  
"mappings" : {  
"_default_" : {  
"\_all" : {"enabled" : true, "omit\_norms" : true},  
"dynamic\_templates" : [ {  
"message\_field" : {  
"match" : "message",  
"match\_mapping\_type" : "string",  
"mapping" : {  
"type" : "string", "index" : "analyzed", "omit\_norms" : true,  
"fielddata" : { "format" : "disabled" }  
}  
}  
}, {  
"string\_fields" : {  
"match" : "\*",  
"match\_mapping\_type" : "string",  
"mapping" : {  
"type" : "string", "index" : "analyzed", "omit\_norms" : true,  
"fielddata" : { "format" : "disabled" },  
"fields" : {  
"raw" : {"type": "string", "index" : "not\_analyzed", "ignore\_above" : 256}  
}  
}  
}  
} ],  
"properties" : {  
"@timestamp": { "type": "date" },  
"@version": { "type": "string", "index": "not\_analyzed" },  
"geoip" : {  
"dynamic": true,  
"properties" : {  
"ip": { "type": "ip" },  
"location" : { "type" : "geo\_point" },  
"latitude" : { "type" : "float" },  
"longitude" : { "type" : "float" }  
}  
}  
"src\_geoip" : {  
"type" : "object",  
"dynamic": true,  
"path": "full",  
"properties" : {  
"location" : { "type" : "geo\_point" }  
}  
}  
"dst\_geoip" : {  
"type" : "object",  
"dynamic": true,  
"path": "full",  
"properties" : {  
"location" : { "type" : "geo\_point" }  
}  
}  
}  
}  
}  
}

**Logstash configuration - Breaking apart src and dst geoip targets.**

geoip {  
source =\> "srcip"  
database =\> "/etc/logstash/geo/custom\_geoip.dat"  
target =\> "src\_geoip"  
}  
geoip {  
source =\> "dstip"  
database =\> "/etc/logstash/geo/custom\_geoip.dat"  
target =\> "dst\_geoip"  
}

Thanks again!  
Cody

---

<div class="post-metadata">

**Author:** ![vikas\_gopal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vikas_gopal/32/47661_2.png) [@vikas\_gopal](https://discuss.elastic.co/u/vikas_gopal)\
**Post date:** [July 15, 2016, 3:09pm UTC](https://discuss.elastic.co/t/can-we-create-two-geoip-filters-in-one-logstash-config-file/42849/13 "2016-07-15T15:09:38Z")

</div>

I can suggest couple of things

1. Not sure if ES 2.3 support "path": "full" for geoip anymore((remove "path": "full" )), so you just need to do something like following in your template

"dst\_geoip" : {"type" : "object","dynamic": true,"properties" : {"location" : { "type" : "geo\_point" }}},  
"src\_geoip" : {"type" : "object","dynamic": true,"properties" : {"location" : { "type" : "geo\_point" }}}

1. When I was using database in logstash 1.5.4 I used following, you need to check if it works for latest LS

geoip {  
source =\> "src"  
target =\> "geoip"  
database =\> "E:\Geo\_database\11-02-2016\GeoLiteCity.dat"  
add\_field =\> ["[geoip][coordinates]", "%{[geoip][longitude]}" ]  
add\_field =\> ["[geoip][coordinates]", "%{[geoip][latitude]}" ]  
}  
mutate {  
convert =\> ["[geoip][coordinates]", "float"]  
}

1. Make sure you are using template\_overwrite =\> "true" in LS

Let me know if it works also check this post [Problems with geoip configuration](https://discuss.elastic.co/t/problems-with-geoip-configuration/28767/25)

---

<div class="post-metadata">

**Author:** ![Cody\_Betsworth](https://avatars.discourse-cdn.com/v4/letter/c/57b2e6/32.png) [@Cody\_Betsworth](https://discuss.elastic.co/u/Cody_Betsworth)\
**Post date:** [July 15, 2016, 3:39pm UTC](https://discuss.elastic.co/t/can-we-create-two-geoip-filters-in-one-logstash-config-file/42849/14 "2016-07-15T15:39:52Z")

</div>

geoip {  
source =\> "srcip"  
database =\> "/etc/logstash/geo/custom\_geoip.dat"  
target =\> "src\_geoip"  
add\_field =\> ["[src\_geoip][coordinates]", "%{[src\_geoip][longitude]}" ]  
add\_field =\> ["[src\_geoip][coordinates]", "%{[src\_geoip][latitude]}" ]  
}  
mutate {  
convert =\> ["[src\_geoip][coordinates]", "float"]  
}  
geoip {  
source =\> "dstip"  
database =\> "/etc/logstash/geo/custom\_geoip.dat"  
target =\> "dst\_geoip"  
add\_field =\> ["[dst\_geoip][coordinates]", "%{[dst\_geoip][longitude]}" ]  
add\_field =\> ["[dst\_geoip][coordinates]", "%{[dst\_geoip][latitude]}" ]  
}  
mutate {  
convert =\> ["[dst\_geoip][coordinates]", "float"]  
}

**Tried to split them this way again. I had to do this also with previous versions of elasticsearch. Still receive the same result. Here is the geoip configuration with the changes. src\_geoip coordinates and dst\_geoip coordinates still show up as a number are not defined as geopoint on the mapping.**

output {  
elasticsearch {  
hosts =\> "10.x.x.x"  
index =\> "logstash-fortinet-%{+YYYY.MM.dd}"  
template\_name =\> "fortinet\*"  
template =\> "/etc/logstash/templates/elasticsearch-fortinet.json"  
manage\_template =\> "true"  
template\_overwrite =\> "true"  
}  
}

**Template mapping**

{  
"template" : "fortinet\*",  
"settings" : {  
"index.refresh\_interval" : "5s"  
},  
"mappings" : {  
"_default_" : {  
"\_all" : {"enabled" : true, "omit\_norms" : true},  
"dynamic\_templates" : [ {  
"message\_field" : {  
"match" : "message",  
"match\_mapping\_type" : "string",  
"mapping" : {  
"type" : "string", "index" : "analyzed", "omit\_norms" : true,  
"fielddata" : { "format" : "disabled" }  
}  
}  
}, {  
"string\_fields" : {  
"match" : "\*",  
"match\_mapping\_type" : "string",  
"mapping" : {  
"type" : "string", "index" : "analyzed", "omit\_norms" : true,  
"fielddata" : { "format" : "disabled" },  
"fields" : {  
"raw" : {"type": "string", "index" : "not\_analyzed", "ignore\_above" : 256}  
}  
}  
}  
} ],  
"properties" : {  
"@timestamp": { "type": "date" },  
"@version": { "type": "string", "index": "not\_analyzed" },  
"geoip" : {  
"dynamic": true,  
"properties" : {  
"ip": { "type": "ip" },  
"location" : { "type" : "geo\_point" },  
"latitude" : { "type" : "float" },  
"longitude" : { "type" : "float" }  
}  
},  
"dst\_geoip" : {"type" : "object","dynamic": true,"properties" : {"location" : { "type" : "geo\_point" }}},  
"src\_geoip" : {"type" : "object","dynamic": true,"properties" : {"location" : { "type" : "geo\_point" }}}  
}  
}  
}  
}

---

<div class="post-metadata">

**Author:** ![Cody\_Betsworth](https://avatars.discourse-cdn.com/v4/letter/c/57b2e6/32.png) [@Cody\_Betsworth](https://discuss.elastic.co/u/Cody_Betsworth)\
**Post date:** [July 15, 2016, 3:42pm UTC](https://discuss.elastic.co/t/can-we-create-two-geoip-filters-in-one-logstash-config-file/42849/15 "2016-07-15T15:42:49Z")

</div>

I can get 1 IP geoip filtering to work quite easily but something about overriding this template just has me stumped. Nothing seems to change it in the mapping even with overrides and creating the manual mapping in the json file. Like you I wanted to be able to geohash Tile Map both dst and src addresses.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [July 15, 2016, 4:09pm UTC](https://discuss.elastic.co/t/can-we-create-two-geoip-filters-in-one-logstash-config-file/42849/16 "2016-07-15T16:09:33Z")

</div>

You _are_ creating new indexes after each template update, right? Not expecting existing indexes to change their mappings?

---

<div class="post-metadata">

**Author:** ![Cody\_Betsworth](https://avatars.discourse-cdn.com/v4/letter/c/57b2e6/32.png) [@Cody\_Betsworth](https://discuss.elastic.co/u/Cody_Betsworth)\
**Post date:** [July 15, 2016, 4:10pm UTC](https://discuss.elastic.co/t/can-we-create-two-geoip-filters-in-one-logstash-config-file/42849/17 "2016-07-15T16:10:48Z")

</div>

Correct. I am clearing the index and just starting fresh each time I change the template. This is a fresh install and I don't plan to start retaining data until I can resolve this issue.

curl -XDELETE [http://10.x.x.x:9200/\*](http://10.x.x.x:9200/*)

---

<div class="post-metadata">

**Author:** ![Cody\_Betsworth](https://avatars.discourse-cdn.com/v4/letter/c/57b2e6/32.png) [@Cody\_Betsworth](https://discuss.elastic.co/u/Cody_Betsworth)\
**Post date:** [July 15, 2016, 4:18pm UTC](https://discuss.elastic.co/t/can-we-create-two-geoip-filters-in-one-logstash-config-file/42849/18 "2016-07-15T16:18:57Z")

</div>

I got it. Thank you both so much for the help. It was a syntax error on my end. The template name in the logstash configuration did not match the template name identified in the mapping.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:47am UTC](https://discuss.elastic.co/t/can-we-create-two-geoip-filters-in-one-logstash-config-file/42849/19 "2017-07-06T04:47:48Z")

</div>