Uknown error while parsing user agent data

Elastic Stack Logstash
1

I'm trying to parse some iis logs and i'm seeing this error in my logstash logs, which is leading me to lot of _grokparsefailure error any idea how to mitigate this

Logstash Logs
[2017-06-25T09:49:02,075][ERROR][logstash.filters.useragent] Uknown error while parsing user agent data {:exception=>java.lang.IllegalStateException: No match found, :field=>"useragent", :event=>2017-06-20T23:25:18.000Z somehost.lan 2017-06-20 23:25:18 172.17.1.1 GET /bootstrap-3.3.7-dist/css/home2.css - 80 - 172.17.1.5 Mozilla/5.0+(Linux;+U;+Android+6.0;+en-US;+Lenovo+A7700+Build/MRA58K)+AppleWebKit/534.30+(KHTML,+like+Gecko)+Version/4.0+UCBrowser/10.10.8.820+U3/0.8.0+Mobile+Safari/534.30 304 0 0 312
}
[2017-06-25T09:49:02,232][ERROR][logstash.filters.useragent] Uknown error while parsing user agent data {:exception=>java.lang.StringIndexOutOfBoundsException: String index out of range: 74, :field=>"useragent", :event=>2017-06-21T01:44:43.000Z somehost.lan 2017-06-21 01:44:43 172.17.1.1 GET /favicon.ico - 80 - 172.17.1.5 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64;+Trident/7.0;+rv:11.0)+like+Gecko 404 0 2 234

Rubydebug
{
"http_response" => "200",
"minor" => "0",
"useragent" => "Mozilla/5.0+(Windows+NT+6.1;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/58.0.3029.110+Safari/537.36",
"type" => "iislogs",
"patch" => "3029",
"path" => "/Users/someuser/Dev/elastic/LogFiles/W3SVC7/u_ex170624.log",
"sc_status" => "0",
"http_method" => "GET",
"major" => "58",
"@version" => "1",
"host" => "somehost.lan",
"os" => "Windows",
"message" => "2017-06-24 18:04:24 ::1 GET /barcode/RD24.jpg - 89 - ::1 Mozilla/5.0+(Windows+NT+6.1;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/58.0.3029.110+Safari/537.36 200 0 0 2\r",
"sub_response" => "0",
"time_taken" => "2",
"site" => "::1",
"@timestamp" => 2017-06-24T12:34:24.000Z,
"port" => "89",
"build" => "",
"name" => "Chrome",
"os_name" => "Windows",
"page" => "/barcode/RD24.jpg",
"client_host" => "::1",
"device" => "Other",
"query_string" => "-",
"username" => "-"
}

1 Like
2

What does your Logstash config look like?

3

Here you go...

input {
file {
type => "iislogs"
path => "/Users/someuser/Dev/elastic/LogFiles/W3SVC*/*.log"
start_position => "beginning"
sincedb_path => "/dev/null"
}
}

filter {

# ignore log comments
if [message] =~ "^#" {
	drop {}
}

# check that fields match your IIS log settings
grok {
	match => ["message", "%{TIMESTAMP_ISO8601:log_timestamp} %{IPORHOST:site} %{WORD:http_method} %{URIPATH:page} %{NOTSPACE:query_string} %{NUMBER:port} %{NOTSPACE:username} %{IPORHOST:client_host} %{NOTSPACE:useragent} %{NUMBER:http_response} %{NUMBER:sub_response} %{NUMBER:sc_status} %{NUMBER:time_taken}"]
}

# set the event timestamp from the log
# https://www.elastic.co/guide/en/logstash/current/plugins-filters-date.html
date {
	match => [ "log_timestamp", "YYYY-MM-dd HH:mm:ss" ]
	timezone => "Asia/Kolkata"
}

# matches the big, long nasty useragent string to the actual browser name, version, etc
# https://www.elastic.co/guide/en/logstash/current/plugins-filters-useragent.html
useragent {
	source=> "useragent"
	prefix=> "browser_"
}

mutate {
	remove_field => [ "log_timestamp"]
}

mutate {
    add_field => { "token" => "GKRUZblvxHvyaLUybRfakTRIzNAIgwDE" }
}

}

output logs to console and to elasticsearch

output {
stdout { codec => rubydebug }
elasticsearch {
hosts => "localhost:9200"
index => "logstash-%{type}-%{+YYYY}"
}
if "_grokparsefailure" in [tags] {
stdout { codec => "rubydebug" }
}
}

4

Which version of Logstash are you using?

5

bin ./logstash --version
logstash 5.4.2

6

Any luck with the issue, while i waited for an answer i even tried to run this same config through logstash 2.4 still the same problem, looks like there is some problem in parsing the user agent info.

7

I got the same error, updating useragent plugin has helped me. Read more here: https://github.com/logstash-plugins/logstash-filter-useragent/pull/46

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.