# Can't get GeoIp data in Elasticsearch/Fluentd

**URL:** <https://discuss.elastic.co/t/cant-get-geoip-data-in-elasticsearch-fluentd/140500>\
**Category:** Elasticsearch\
**Created:** [July 18, 2018, 9:49am UTC](https://discuss.elastic.co/t/cant-get-geoip-data-in-elasticsearch-fluentd/140500 "2018-07-18T09:49:18Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![mprzygrodzki](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mprzygrodzki/32/34738_2.png) [@mprzygrodzki](https://discuss.elastic.co/u/mprzygrodzki)\
**Post date:** [July 18, 2018, 9:49am UTC](https://discuss.elastic.co/t/cant-get-geoip-data-in-elasticsearch-fluentd/140500/1 "2018-07-18T09:49:18Z")

</div>

Hello, i'm using Fluentd to push logs into Elasticsearch. The main goal is to have nginx-ingress logs published into Elasticsearch with geoip location so I can visualize metrics on dashboard with map based on IP's. I'm using fluent-plugin-geoip-1.2.0 as exporter for this kind of data. I'm already publishing nginx-ingress logs to Elasticsearch in json format and it works as it should, i can search the data in Kibana output looks like below:

`{"proxy_protocol_addr": "89.72.XXX.XXX","remote_addr": "89.72.XXX.XXX", "proxy_add_x_forwarded_for": "89.72.110.107", "request_id": "8a49870b2cee49911b0793ec97226036","remote_user": "", "time_local": "17/Jul/2018:20:42:15 +0000", "request" : "GET /app/kibana HTTP/1.1", "status": "200", "vhost": "kibana.staging.domain.com","body_bytes_sent": "14225", "http_referer": "http://kibana.staging.domain.com/", "http_user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36", "request_length" : "480", "request_time" : "0.246", "proxy_upstream_name": "monitoring-kibana-80", "upstream_addr": "100.100.72.251:5601", "upstream_response_length": "14196", "upstream_response_time": "0.244", "upstream_status": "200"}`

**Here's my config for geoip data:**

```
<filter kubernetes.**>
    @type geoip
    # Specify one or more geoip lookup field which has ip address (default: host)
    # in the case of accessing nested value, delimit keys by dot like 'host.ip'.
    geoip_lookup_keys remote_addr
    # Specify optional geoip database (using bundled GeoLiteCity databse by default)
    geoip_database "/var/lib/gems/2.3.0/gems/fluent-plugin-geoip-1.2.0/data/GeoLiteCity.dat"
    # Set adding field with placeholder (more than one settings are required.)
    <record>
      city ${city["remote_addr"]}
      lat ${latitude["remote_addr"]}
      lon ${longitude["remote_addr"]}
      country_code3 ${country_code3["remote_addr"]}
      country ${country_code["remote_addr"]}
      country_name ${country_name["remote_addr"]}
      dma ${dma_code["remote_addr"]}
      area ${area_code["remote_addr"]}
      region ${region["remote_addr"]}
      geoip '{"location":[${longitude["remote_addr"]},${latitude["remote_addr"]}]}'
    </record>
    # To avoid get stacktrace error with `[null, null]` array for elasticsearch.
    skip_adding_null_record true
    # Set log_level for fluentd-v0.10.43 or earlier (default: warn)
    @log_level info
    # Set buffering time (default: 0s)
    # flush_interval 1s
</filter>

```

**I've checked mappings using "GET /\_template" and "GET /\_template/logstash-\*" and in both cases i see mapping are created like below**

```
"geoip": {
            "dynamic": true,
            "properties": {
              "ip": {
                "type": "ip"
              },
              "location": {
                "type": "geo_point"
              },
              "latitude": {
                "type": "half_float"
              },
              "longitude": {
                "type": "half_float"
              }

```

**What's wrong or what i'm missing to get this working as expected, and see geoip data ?**

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [July 20, 2018, 12:07am UTC](https://discuss.elastic.co/t/cant-get-geoip-data-in-elasticsearch-fluentd/140500/2 "2018-07-20T00:07:26Z")

</div>

I don't know fluentd, but what does the output document it creates look like?

---

<div class="post-metadata">

**Author:** ![mprzygrodzki](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mprzygrodzki/32/34738_2.png) [@mprzygrodzki](https://discuss.elastic.co/u/mprzygrodzki)\
**Post date:** [July 20, 2018, 9:32am UTC](https://discuss.elastic.co/t/cant-get-geoip-data-in-elasticsearch-fluentd/140500/3 "2018-07-20T09:32:26Z")

</div>

I believe i'm missing something in output.

```
output.conf: |
    # Enriches records with Kubernetes metadata
    <filter kubernetes.**>
      @type kubernetes_metadata
    </filter>
    <filter **>
      @type grep
      <exclude>
        key log
        pattern ElasticsearchErrorHandler
      </exclude>  
    </filter>    

<match **>
  @id elasticsearch
  @type elasticsearch
  @log_level info
  include_tag_key true
  host "#{ENV['OUTPUT_HOST']}"
  port "#{ENV['OUTPUT_PORT']}"
  logstash_format true
  <buffer>
    @type file
    path /var/log/fluentd-buffers/kubernetes.system.buffer
    flush_mode interval
    retry_type exponential_backoff
    flush_thread_count 2
    flush_interval 5s
    retry_forever
    retry_max_interval 30
    chunk_limit_size "#{ENV['OUTPUT_BUFFER_CHUNK_LIMIT']}"
    queue_limit_length "#{ENV['OUTPUT_BUFFER_QUEUE_LIMIT']}"
    overflow_action block
  </buffer>
</match>
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 17, 2018, 9:32am UTC](https://discuss.elastic.co/t/cant-get-geoip-data-in-elasticsearch-fluentd/140500/4 "2018-08-17T09:32:30Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.