# Logstash AWS GeoIP Issue? **URL:** \ **Category:** Logstash\ **Created:** [September 14, 2015, 6:40pm UTC](https://discuss.elastic.co/t/logstash-aws-geoip-issue/29274 "2015-09-14T18:40:18Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![russK](https://avatars.discourse-cdn.com/v4/letter/r/9fc348/32.png) [@russK](https://discuss.elastic.co/u/russK)\ **Post date:** [September 14, 2015, 6:40pm UTC](https://discuss.elastic.co/t/logstash-aws-geoip-issue/29274/1 "2015-09-14T18:40:18Z")
Hello, We're running an ELK cluster on AWS. We also run a single box with all of ELK on it to verify changes locally before pushing to the cloud. We're running into an issue where the local box can run the GeoIP2-City.mmdb database and pull all of the information out correctly. When we try to do the same thing on AWS we're getting errors. Specifically, here's what we're seeing: > Unknown error while looking up GeoIP data {:exception=\>#\, :field=\>nil, :event=\>#\"2015-09-14T15:19:34.328933Z clusterID 66.109.145.98:56225 10.0.1.128:6443 0.000047 0.09505 0.000035 200 200 0 6527 GET [https://clusterInfo](https://clusterInfo)", "@version"=\>"1", "@timestamp"=\>"2015-09-14T17:44:28.805Z", "host"=\>"10.0.0.102:55794", "type"=\>"ELB", "tags"=\>["\_grokparsefailure"]}, @lut={"host"=\>[{"message"=\>"2015-09-14T15:19:34.328933Z clusterID 66.109.145.98:56225 10.0.1.128:6443 0.000047 0.09505 0.000035 200 200 0 6527 GET [https://clusterInfo](https://clusterInfo)", "@version"=\>"1", "@timestamp"=\>"2015-09-14T17:44:28.805Z", "host"=\>"10.0.0.102:55794", "type"=\>"ELB", "tags"=\>["\_grokparsefailure"]}, "host"], "type"=\>[{"message"=\>"2015-09-14T15:19:34.328933Z clusterID 66.109.145.98:56225 10.0.1.128:6443 0.000047 0.09505 0.000035 200 200 0 6527 GET [https://clusterInfo](https://clusterInfo)", "@version"=\>"1", "@timestamp"=\>"2015-09-14T17:44:28.805Z", "host"=\>"10.0.0.102:55794", "type"=\>"ELB", "tags"=\>["\_grokparsefailure"]}, "type"], "[type]"=\>[{"message"=\>"2015-09-14T15:19:34.328933Z clusterID 66.109.145.98:56225 10.0.1.128:6443 0.000047 0.09505 0.000035 200 200 0 6527 GET [https://clusterInfo](https://clusterInfo)", "@version"=\>"1", "@timestamp"=\>"2015-09-14T17:44:28.805Z", "host"=\>"10.0.0.102:55794", "type"=\>"ELB", "tags"=\>["\_grokparsefailure"]}, "type"], "message"=\>[{"message"=\>"2015-09-14T15:19:34.328933Z clusterID 66.109.145.98:56225 10.0.1.128:6443 0.000047 0.09505 0.000035 200 200 0 6527 GET [https://clusterInfo](https://clusterInfo)", "@version"=\>"1", "@timestamp"=\>"2015-09-14T17:44:28.805Z", "host"=\>"10.0.0.102:55794", "type"=\>"ELB", "tags"=\>["\_grokparsefailure"]}, "message"], "tags"=\>[{"message"=\>"2015-09-14T15:19:34.328933Z clusterID 66.109.145.98:56225 10.0.1.128:6443 0.000047 0.09505 0.000035 200 200 0 6527 GET [https://clusterInfo](https://clusterInfo)", "@version"=\>"1", "@timestamp"=\>"2015-09-14T17:44:28.805Z", "host"=\>"10.0.0.102:55794", "type"=\>"ELB", "tags"=\>["\_grokparsefailure"]}, "tags"], "client\_ip"=\>[{"message"=\>"2015-09-14T15:19:34.328933Z clusterID 66.109.145.98:56225 10.0.1.128:6443 0.000047 0.09505 0.000035 200 200 0 6527 GET [https://clusterInfo](https://clusterInfo)", "@version"=\>"1", "@timestamp"=\>"2015-09-14T17:44:28.805Z", "host"=\>"10.0.0.102:55794", "type"=\>"ELB", "tags"=\>["\_grokparsefailure"]}, "client\_ip"]}\>, @data={"message"=\>"2015-09-14T15:19:34.328933Z clusterID 66.109.145.98:56225 10.0.1.128:6443 0.000047 0.09505 0.000035 200 200 0 6527 GET [https://clusterInfo](https://clusterInfo)", "@version"=\>"1", "@timestamp"=\>"2015-09-14T17:44:28.805Z", "host"=\>"10.0.0.102:55794", "type"=\>"ELB", "tags"=\>["\_grokparsefailure"]}, @cancelled=false\>, :level=\>:error} We have the exact same db, OS version, ELK versions, and logstash.conf (except where output points) file running on the two ELK platforms - local and cloud. We also stood up another ELK box on AWS to see if having all of ELK on one box somehow prevented the issue, but alas it's also having the same problem with Logstash as the other AWS device. Here's our logstash.conf file: Has to be attached in second post. Any ideas on how to fix this? ---
**Author:** ![russK](https://avatars.discourse-cdn.com/v4/letter/r/9fc348/32.png) [@russK](https://discuss.elastic.co/u/russK)\ **Post date:** [September 14, 2015, 6:41pm UTC](https://discuss.elastic.co/t/logstash-aws-geoip-issue/29274/2 "2015-09-14T18:41:19Z")
As mentioned in the first post here's a copy of our logstash.conf file. > input { > tcp{ > port =\> 9292 > } > tcp{ > port =\> 9230 > type =\> ELB > ``` > } > tcp{ > port => 9240 > type => ArcGIS > } > udp{ > port => 9292 > } > > ``` > > } > filter{ > if [type] == "ELB"{ > grok{ > match =\> {"message" =\> "%{TIMESTAMP\_ISO8601:timestamp} %{NOTSPACE:loadbalancer} %{IP:client\_ip}:%{NUMBER:client\_port:int} %{IP:backend\_ip}:%{NUMBER:backend\_port:int} %{NUMBER:request\_processing\_time:float} %{NUMBER:backend\_processing\_time:float} %{NUMBER:response\_processing\_time:float} %{NUMBER:elb\_status\_code:int} %{NUMBER:backend\_status\_code:int} %{NUMBER:received\_bytes:int} %{NUMBER:sent\_bytes:int} %{WORD:verb} %{NOTSPACE:request} HTTP/%{NUMBER:httpversion} %{NOTSPACE:user\_agent} %{NOTSPACE:ssl\_cipher} %{NOTSPACE:ssl\_protocol}" > } > } > geoip { > source =\> "client\_ip" > target =\> "geoip" > database =\> "/etc/logstash/GeoIP2-City.mmdb" > add\_field =\> ["[geoip][coordinates]", "%{[geoip][longitude]}" ] > add\_field =\> ["[geoip][coordinates]", "%{[geoip][latitude]}" ] > } > mutate { > convert =\> ["[geoip][coordinates]", "float"] > } > } > ``` > if [type] == "ArcGIS"{ > grok{ > match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{NOTSPACE:LogType} %{NUMBER:code} %{NOTSPACE:target} %{NOTSPACE:machine} %{NUMBER:process} %{NUMBER:thread} %{NOTSPACE:user} %{NUMBER:XMin} %{NUMBER:YMin} %{NUMBER:XMax} %{NUMBER:YMax} %{NUMBER:XCent} %{NUMBER:YCent} %{NUMBER:SizeOne} %{NUMBER:SizeTwo} %{NUMBER:Scale}" > } > } > > ``` > ``` > if [XCent] and [YCent] { > mutate { > add_field => ["[userMapXY]", "%{XCent}" ] > add_field => ["[userMapXY]", "%{YCent}" ] > } > mutate { > convert => ["[userMapXY]", "float" ] > } > } > } > > ``` > > } > output { > elasticsearch { > host =\> "localhost:9200" > protocol =\> "http" > } > } ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [September 14, 2015, 7:39pm UTC](https://discuss.elastic.co/t/logstash-aws-geoip-issue/29274/3 "2015-09-14T19:39:37Z")
The records that failed have a "\_grokparsefailure" tag set, which indicates that the grok parsing failed and that the client\_ip field, which the geoip filter relies on, therefore will was be extracted. Are you feeding different data in the two systems? ---
**Author:** ![russK](https://avatars.discourse-cdn.com/v4/letter/r/9fc348/32.png) [@russK](https://discuss.elastic.co/u/russK)\ **Post date:** [September 14, 2015, 8:41pm UTC](https://discuss.elastic.co/t/logstash-aws-geoip-issue/29274/4 "2015-09-14T20:41:08Z")
The data is fed the exact same way in the two systems. Also by changing the DB from a dot mmdb to the dot dat within the logstash config AWS will happily parse all of the data. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [September 14, 2015, 8:59pm UTC](https://discuss.elastic.co/t/logstash-aws-geoip-issue/29274/5 "2015-09-14T20:59:24Z")
> [@russK](#): > > GeoIP2-City.mmdb Is that the latest release of the Maxmind DB? Cause we only support the original, but now deprecated version, not the newest release. Please comment on [this GH issue](https://github.com/logstash-plugins/logstash-filter-geoip/issues/23) if you'd like to see support for the newer version of the database. ---
**Author:** ![russK](https://avatars.discourse-cdn.com/v4/letter/r/9fc348/32.png) [@russK](https://discuss.elastic.co/u/russK)\ **Post date:** [September 14, 2015, 9:13pm UTC](https://discuss.elastic.co/t/logstash-aws-geoip-issue/29274/6 "2015-09-14T21:13:23Z")
Yes, that's the proprietary version of the db. What's interesting is that it works just fine on a local machine, but won't work on AWS. Is there some difference with running logstash on the cloud that would cause that to be an issue? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [September 14, 2015, 9:23pm UTC](https://discuss.elastic.co/t/logstash-aws-geoip-issue/29274/7 "2015-09-14T21:23:37Z")
What version of LS? Also is it the proprietary one, or the newer version of the "free" one? ---
**Author:** ![russK](https://avatars.discourse-cdn.com/v4/letter/r/9fc348/32.png) [@russK](https://discuss.elastic.co/u/russK)\ **Post date:** [September 14, 2015, 10:46pm UTC](https://discuss.elastic.co/t/logstash-aws-geoip-issue/29274/8 "2015-09-14T22:46:11Z")
Logstash 1.4.2 This is a copy of the pay for version of the max mind database. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 5:29am UTC](https://discuss.elastic.co/t/logstash-aws-geoip-issue/29274/9 "2017-07-06T05:29:09Z")