Logstash filter conditions does not work,how to get The real user ipaddress from access log of Apache?

Elastic Stack Logstash
1

Logstash filter conditions does not work， as follows：

filter {
      json {
            source => "message"
        }

        if "%{client_ip}" == "-" {
            mutate {
                remove_field => "client_ip"
                add_field => { "client_ip" => "0.0.0.0" }
            }
        }
}

when %{client_ip} == "-" I want replace it to 0.0.0.0 ，but it does not work，still output like ：

{
              "method" => "GET",
     "body_bytes_sent" => "182",
              "source" => "/data/wwwlogs/access_20170920.log",
             "message" => "{\"@timestamp\":\"2017-09-20T12:29:01+0800\",\"client_ip\":\"-\",\"request_time\":0,\"status\":200,\"url\":\"/web/123.gif\",\"method\":\"GET\",\"http_host\":\"www.test.com\",\"server_ip\":\"192.168.1.10\",\"http_referer\":\"-\",\"http_user_agent\":\"Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.86 Safari/537.36\",\"body_bytes_sent\":\"182\",\"total_bytes_sent\":\"482\"}",
           "http_host" => "www.test.com",
                 "url" => "/web/123.gif",
     "http_user_agent" => "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.86 Safari/537.36",
          "@timestamp" => 2017-09-20T04:29:01.000Z,
        "request_time" => 0,
        "http_referer" => "-",
    "total_bytes_sent" => "482",
                "beat" => {},
           "server_ip" => "192.168.1.10",
           "client_ip" => "-",
              "status" => 200
}
2

Use if [client_ip] ... instead. See https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html.

3

it has resolved!

if [client_ip] == "-" {
     replace => { "client_ip" => "0.0.0.0" }
}
4

Tks :grin:

5

For get the real client ip, we could editor the configuration of Apache and logstash like this :

Apache

#add  client_ip and direct_ip

LogFormat "{ \
           \"@timestamp\": \"%{%Y-%m-%dT%H:%M:%S%z}t\", \

           \"client_ip\": \"%{X-Forwarded-For}i\", \
           \"direct_ip\": \"%a\", \
          
            \"request_time\": %T, \
           \"status\": %>s, \
           \"url\": \"%U%q\", \
           \"method\": \"%m\", \
           \"http_host\": \"%{Host}i\", \
           \"server_ip\": \"%A\", \
           \"http_referer\": \"%{Referer}i\", \
           \"http_user_agent\": \"%{User-agent}i\", \
	   \"body_bytes_sent\": \"%B\", \
	   \"total_bytes_sent\": \"%O\" \
 }"  access_log_json

CustomLog "|/usr/local/apache2/bin/rotatelogs -l /data/wwwlogs/access_%Y%m%d.log 86400" access_log_json

logstash filter plugin：

mutate {  
      json {
            source => "message"
        }
        mutate {
            split => ["client_ip", ","]
        }

        mutate {
            replace => { "client_ip" => "%{client_ip[0]}" }
        }

        if [client_ip] == "-" {
                mutate {
                    replace => { "client_ip" => "%{direct_ip}" }
                }
        }
        mutate {
            remove_field => "direct_ip"
        }

    }
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.