Logstash Error "Error: Address already in use"

Elastic Stack Logstash
1

Hi,
I have apache pipeline with multiple match patterns. Here is the code

input {
beats {
port => 5044
id => "apache-access"
}
}
filter {
if [fields][log] == "apache-access" {
grok {
match => [
"message", '%{IPORHOST:clientip} %{USER:ident} %{USER:auth} [%{HTTPDATE:timestamp}] "%{WORD:verb} %{DATA:request} HTTP/%{NUMBER:httpversion}" %{NUMBER:response:int} (?:-|%{NUMBER:bytes:int}) %{QS:referrer} %{QS:agent}',
"message", '%{IPORHOST:clientip} %{USER:ident} %{USER:auth} [%{HTTPDATE:timestamp}] "%{WORD:verb} %{DATA:request} HTTP/%{NUMBER:httpversion}" %{NUMBER:response:int} (?:-|%{NUMBER:bytes:int}) (?:-|%{NUMBER:tosinsec:int}) (?:-|%{NUMBER:tosinmicrosec:int})'
]
}
}

When i run with "--config.test_and_exit" flag, it says "Config Validation Result: OK". But when i start logstash service i am seeing this error in logstash log file

Error: Address already in use
Exception: Java::JavaNet::BindException

2

I am trying to write one pipeline to parse the below logs

127.0.0.1 - - [09/Nov/2018:10:58:28 +0530] "GET / HTTP/1.1" 403 4897 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0"
127.0.0.1 - - [09/Nov/2018:10:58:28 +0530] "GET test.min.css HTTP/1.1" 200 19341 "http://127.0.0.1/" "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0"
127.0.0.1 - - [17/Jan/2019:11:43:26 +0530] "GET /theme/css/test.css HTTP/1.1" 200 1170 0 285
127.0.0.1 - - [17/Jan/2019:11:43:26 +0530] "GET /theme/css/test.css HTTP/1.1" 200 314 0 240

some logs have "referer & user-agent" and some doesn't have. Other logs have extra data like "time to serve in sec and milliseconds".

Please tell me how to write one pipeline with multiple matches to work for both types of logs.

3

I would use dissect to do the initial parsing

    dissect { mapping => { "message" => '%{clientIP} - - [%{ts}] "%{method} %{uri} %{protocol}" %{status} %{something} %{restOfLine}' } }

Then use a grok with multiple patterns to do the rest

grok {
    match => {
        "restOfLine" => [
            "^%{INT:i1} %{INT:i2}",
            "^%{QS:referer} %{QS:useragent}"
        ]
    }
}
4

Thanks Badger. I will try your solution. Is there is anything wrong with my code?

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.