GeoIP - can't get it working

Elastic Stack Logstash
1

Hi all,

I've followed this tutorial on how to add GeoIP to LS:

DO GeoIP

I've setup my apache logs to output JSON to Logstash. It's working pretty well. And since I added the json codec to the beats plugin, the log entries format nicely.

I have tags setup so I can pull up log entries by these tags:

  • apache_ref_access

  • apache_dev_access

  • apache_beta_access

I've downloaded a GeoIP database and put it here:

[root@logs:~] #ls -l /etc/logstash/data.d/GeoLiteCity.dat
-rw-r--r-- 1 root root 19136630 Feb 12 08:24 /etc/logstash/data.d/GeoLiteCity.dat

And I setup a filter like this:

if [tags] == "apache_ref_access" or [tags] == "apache_dev_access" or [tags] == "apache_beta_access" {

     geoip {
      source => "clientip"
      target => "geoip"
      database => "/etc/logstash/data.d/GeoLiteCity.dat"
      add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ]
      add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}"  ]
    }
    mutate {
      convert => [ "[geoip][coordinates]", "float"]
    }

}

Then restarted logstash. But now when I take a look at my logs, I'm not seeing any GeoIP info:

{
  "_index": "logstash-2016.02.12",
  "_type": "apache_ref_access",
  "_id": "AVLWdB-HTXr1vl4bIsZx",
  "_score": null,
  "_source": {
    "host": "ref.example.com.example.com",
    "path": "/var/log/httpd/jf_ref.example.com_access_log",
    "tags": [
      "example-dev",
      "web-tier",
      "beats",
      "beats_input_codec_json_applied"
    ],
    "message": "213.238.179.240 - - [12/Feb/2016:11:01:03 -0500] \"GET /joke_details.php?joke_id=id277&v=2 HTTP/1.1\" 200 966",
    "timestamp": "2016-02-12T11:01:03-0500",
    "clientip": "213.238.179.240",
    "duration": 1087946,
    "status": 200,
    "request": "/joke_details.php?joke_id=id277&v=2",
    "urlpath": "/joke_details.php",
    "urlquery": "?joke_id=id277&v=2",
    "method": "GET",
    "bytes": 966,
    "vhost": "ref.example.com",
    "@version": "1",
    "@timestamp": "2016-02-12T16:01:24.558Z",
    "beat": {
      "hostname": "web1",
      "name": "filebeat"
    },
    "count": 1,
    "fields": {
      "service": "apache",
      "type": "apache_ref_access"
    },
    "input_type": "log",
    "offset": 11494183,
    "source": "/var/log/httpd/jf_ref.example.com_access_log",
    "type": "apache_ref_access"
  },
  "fields": {
    "@timestamp": [
      1455292884558
    ]
  },
  "highlight": {
    "fields.type": [
      "@kibana-highlighted-field@apache_ref_access@/kibana-highlighted-field@"
    ],
    "type.raw": [
      "@kibana-highlighted-field@apache_ref_access@/kibana-highlighted-field@"
    ],
    "type": [
      "@kibana-highlighted-field@apache_ref_access@/kibana-highlighted-field@"
    ],
    "fields.type.raw": [
      "@kibana-highlighted-field@apache_ref_access@/kibana-highlighted-field@"
    ]
  },
  "sort": [
    1455292884558
  ]
}

Am I missing anything that I would need to get this working? What can I do to make this happen?

Thanks

2

Hello,
nothing looks wrong with your config except that in your if condition you are testing against [tags] were I would expect you to test againts [type] ? so your events simply did not pass by the filter ???

3

Yup!! That was it. Got it now. It works!! Thanks