# Logstash geoip filter not working properly for certain public IPs when pushing data to es **URL:** \ **Category:** Logstash\ **Created:** [March 26, 2020, 8:08pm UTC](https://discuss.elastic.co/t/logstash-geoip-filter-not-working-properly-for-certain-public-ips-when-pushing-data-to-es/225279 "2020-03-26T20:08:34Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![arunporwal](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@arunporwal](https://discuss.elastic.co/u/arunporwal)\ **Post date:** [March 26, 2020, 8:08pm UTC](https://discuss.elastic.co/t/logstash-geoip-filter-not-working-properly-for-certain-public-ips-when-pushing-data-to-es/225279/1 "2020-03-26T20:08:34Z")
I am pushing aws ELB logs to elasticsearch with using below config as: ```auto input { s3 { type => "elb-logs" bucket => "" region => "us-east-1" sincedb_path => "/var/lib/logstash/since.db" } } filter { if [type] == "elb-logs" { grok { match => ["message", "%{TIMESTAMP_ISO8601:timestamp} %{NOTSPACE:elb_name} %{IP:elb_client_ip}:%{INT:elb_client_port:int} (?:%{IP:elb_backend_ip}:%{NUMBER:elb_backend_port:int}|-) %{NUMBER:elb_request_processing_time:float} %{NUMBER:elb_backend_processing_time:float} %{NUMBER:elb_response_processing_time:float} (?:%{INT:elb_status_code:int}|-) (?:%{INT:elb_backend_status_code:int}|-) %{INT:elb_received_bytes:int} %{INT:elb_sent_bytes:int} \"(?:%{GREEDYDATA:elb_request}|-)\" \"(?:%{GREEDYDATA:elb_userAgent}|-)\" %{NOTSPACE:elb_sslcipher} %{NOTSPACE:elb_sslprotocol}"] match => ["message", "%{GREEDYDATA:elb_event_name} for ELB: %{NOTSPACE:elb_name} at %{TIMESTAMP_ISO8601:timestamp}"] } if [elb_request] =~ /.+/ { grok { match => ["elb_request", "(?:%{WORD:elb_http_method}) (?:%{DATA:elb_http_path})? (?:%{DATA:elb_http_type}/%{NUMBER:elb_http_version:float})?|%{GREEDYDATA:rawrequest}"] } } if [elb_http_path] =~ /.+/ { grok { match => ["elb_http_path", "(?:%{WORD:elb_http_path_protocol}://)?(%{NOTSPACE:elb_http_path_site}:)?(?:%{NUMBER:elb_http_path_port:int})?(?:%{GREEDYDATA:elb_http_path_url})?"] } } fingerprint { source => ["message"] target => "[@metadata][fingerprint]" method => "MURMUR3" } geoip { source => "elb_client_ip" fields => ["city_name", "continent_code", "country_code2", "country_code3","country_name", "dma_code", "postal_code", "region_code", "region_name", "location", "latitude", "longitude","region_name","timezone"] } useragent { source => "elb_userAgent" } } } output { if [type] == "elb-logs" { elasticsearch { hosts => [":port"] ilm_enabled => false index => "aws-elblogs-%{+YYYY.MM.dd}" document_id => "%{[@metadata][fingerprint]}" } } } ``` All of the data is pushed fine in elasticsearch with correct mapping, but for some ips it was not giving proper latitude and longitude and also not adding proper fields as : ```auto city_name continent_code country_code2 country_code3 country_name dma_code region_code region_name ``` example logs as : ```auto 2019-11-23T10:30:10.103697Z **2.16.106.29** :44924 172.18.10.94:31700 0.000045 0.015174 0.000044 200 200 0 1893 "GET HTTP/1.1" "Pingdom.com_bot_version_1.4_(http://www.pingdom.com/)" ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 ``` For above log it was giving latitude and longitude as 42 and 8. it is happening for one series of IPs as `2.*.*.*`. All other document were fine. I tried debugging mode with using stdout method, the result was fine with no issue but in es it is not correct. Any help here ? Another example in debug mode : ```auto [2020-03-26T19:32:07,537][DEBUG][logstash.pipeline] filter received {"event"=>{"@version"=>"1", "message"=>"2019-11-11T21:18:39.394297Z 2.18.240.101:51393 172.18.4.102:30207 0.000043 0.003012 0.000026 200 200 0 0 \"HEAD / HTTP/1.1\" \"Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.85 Safari/537.36\" ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2\n", "@timestamp"=>2020-03-26T19:32:06.822Z, "type"=>"elb-logs"}} ``` ```auto [2020-03-26T19:32:07,634][DEBUG][logstash.pipeline] output received {"event"=>{"elb_status_code"=>200, "elb_http_version"=>1.1, "elb_http_method"=>"HEAD", "elb_http_path_port"=>443, "elb_client_ip"=>"2.18.240.101", "os"=>"Windows", "elb_backend_ip"=>"172.18.4.102", "elb_request_processing_time"=>4.3e-05, "patch"=>"2214", "elb_client_port"=>51393, "elb_sslprotocol"=>"TLSv1.2", "elb_http_path_protocol"=>"https", "elb_backend_port"=>30207, "os_name"=>"Windows", "build"=>"", "message"=>"2019-11-11T21:18:39.394297Z 2.18.240.101:51393 172.18.4.102:30207 0.000043 0.003012 0.000026 200 200 0 0 \"HEAD HTTP/1.1\" \"Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.85 Safari/537.36\" ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2\n", "elb_backend_processing_time"=>0.003012, "elb_sent_bytes"=>0, "elb_http_path_url"=>"/", "name"=>"Chrome", "elb_sslcipher"=>"ECDHE-RSA-AES128-GCM-SHA256", "minor"=>"0", "elb_name"=>"", "elb_received_bytes"=>0, "geoip"=>{"timezone"=>"Europe/Vaduz", "latitude"=>47.0, "continent_code"=>"EU", "location"=>{"lat"=>47.0, "lon"=>8.0}, "longitude"=>8.0}, "type"=>"elb-logs", "elb_http_type"=>"HTTP", "elb_response_processing_time"=>2.6e-05, "elb_http_path_site"=>"", "@timestamp"=>2020-03-26T19:32:06.822Z, "elb_http_path"=>"", "@version"=>"1", "timestamp"=>"2019-11-11T21:18:39.394297Z", "elb_userAgent"=>"Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.85 Safari/537.36", "device"=>"Other", "elb_request"=>"HEAD HTTP/1.1", "major"=>"40", "elb_backend_status_code"=>200}} ``` ---
**Author:** ![arunporwal](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@arunporwal](https://discuss.elastic.co/u/arunporwal)\ **Post date:** [March 26, 2020, 9:12pm UTC](https://discuss.elastic.co/t/logstash-geoip-filter-not-working-properly-for-certain-public-ips-when-pushing-data-to-es/225279/2 "2020-03-26T21:12:03Z")
It is resolved. There was some issue in default database. I manually downloaded db and change point to that db. `database => "/etc/logstash/GeoLite2-City.mmdb"` It worked. 🙂 ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 23, 2020, 9:12pm UTC](https://discuss.elastic.co/t/logstash-geoip-filter-not-working-properly-for-certain-public-ips-when-pushing-data-to-es/225279/3 "2020-04-23T21:12:05Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.