Hi,
I'm trying to add a geoip filter.
My configuration file looks like this:
input {
beats {
port => "5044"
}
}
filter {
grok{
match => {
message => "%{TIMESTAMP_ISO8601:timestamp}%{SPACE}[%{DATA:thread}]%{SPACE}%{LOGLEVEL:loglevel}%{SPACE}appName=%{DATA:appName}%{SPACE}globalId=%{UUID:globalId}%{SPACE}%{JAVACLASS:class}->%{JAVAMETHOD:method}:%{NUMBER:line}%{SPACE}-%{SPACE}%{JAVALOGMESSAGE:uri}|%{JAVALOGMESSAGE:response}|%{IPORHOST:clientip}"
}
add_field => {
"logType" => "message"
}
}
}
filter {
geoip {
source => "clientip"
target => "geoip"
}
}
output {
if [logType] == "message" {
elasticsearch {
hosts => [ "localhost:9200" ]
index => ["service-logs-messages-%{+YYYY.MM.dd}"]
}
}
}
And a sample log line looks like this:
2018-08-07 04:52:59.250 [qtp416049054-72] DEBUG appName=rights globalId=5608ddfb-dca1-43e3-aa42-7877346b4342 org.eclipse.jetty.util.thread.QueuedThreadPool->run:590 - GET//smukvdev01:8086/season/ID/123|200OK|10.190.30.13
When I view the index, I get the following fields:
{
"_index" : "service-logs-messages-2018.08.28",
"_type" : "doc",
"_id" : "3Hh6gGUBsGhNvNlJ4Iyh",
"_score" : 1.0,
"_source" : {
"globalId" : "5608ddfb-dca1-43e3-aa42-7877346b4342",
"method" : "run",
"beat" : {
"name" : "SMUKVDEV01.sintecmedia.com",
"version" : "6.3.2",
"hostname" : "SMUKVDEV01.sintecmedia.com"
},
"thread" : "qtp416049054-72",
"clientip" : "10.190.30.13",
"prospector" : {
"type" : "log"
},
"input" : {
"type" : "log"
},
"message" : "2018-08-07 04:52:59.250 [qtp416049054-72] DEBUG appName=rights globalId=5608ddfb-dca1-43e3-aa42-7877346b4342 org.eclipse.jetty.util.thread.QueuedThreadPool->run:590 - GET//smukvdev01:8086/season/ID/123|200OK|10.190.30.13",
"timestamp" : "2018-08-07 04:52:59.250",
"logType" : "message",
"tbwweekday" : "2",
"uri" : "GET//smukvdev01:8086/season/ID/abc",
"geoip" : { },
"tags" : [
"beats_input_codec_plain_applied",
"_grokparsefailure",
"_dateparsefailure",
"_geoip_lookup_failure"
],
"loglevel" : "DEBUG",
"source" : "/home/elastic/Logs/rights_ira_newFormat.log",
"response" : "200OK",
"logdate" : "2018-08-07T04:52:59.250Z",
"line" : "590",
"host" : {
"name" : "SMUKVDEV01.sintecmedia.com"
},
"@version" : "1",
"@timestamp" : "2018-08-28T12:20:58.575Z",
"offset" : 256,
"appName" : "rights",
"class" : "org.eclipse.jetty.util.thread.QueuedThreadPool"
}
}
I'm getting the _geoip_lookup_failure error and "geoip" is returned empty.
Could you assist with possible solution, please?
Thanks.