How to use Logstash to perform multi-attribute missing tagging on log data?

Elastic Stack Logstash
1

I want to mark log data with missing attributes, but the results always do not match my expectations.
The following is an example of normal log data：
14.49.42.25 - - [12/May/2019:01:24:44 +0000] "GET /articles/ppp-over-ssh/ HTTP/1.1" 200 18586 "-" "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2b1) Gecko/20091014 Firefox/3.6b1 GTB5"
But i want to mark log data with missing attributes,for example:
14.49.42.25 - - [12/May/2019:01:24:44 +0000] "GET /articles/ppp-over-ssh/ HTTP/1.1" 200 "-" "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2b1) Gecko/20091014 Firefox/3.6b1 GTB5",the bytes is missing.
My configuration file is as follows：
input{
stdin{}
}

filter {
grok {
match => { "message" => '%{IPORHOST:clientip} %{USER:ident} %{USER:auth} [%{HTTPDATE:timestamp}] "(?:%{WORD:method} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})" %{NUMBER:response} %{NUMBER:bytes} "(?:%{URI:referrer}|-)" "(?:%{DATA:agent})"' }
}

if [clientip] =="" or ![clientip]{
mutate{
add_field => { "clientip_status" => "missing"}
}
}
if [bytes] == "" or ![bytes]{
mutate{
add_field => { "bytes_status" => "missing"}
}
}
}

output {
stdout { }
}
give the result as follows：
{
"bytes_status" => "missing",
"@version" => "1",
"@timestamp" => 2024-08-28T02:33:59.895177595Z,
"message" => "14.49.42.25 - - [12/May/2019:01:24:44 +0000] "GET /articles/ppp-over-ssh/ HTTP/1.1" 200 "-" "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2b1) Gecko/20091014 Firefox/3.6b1 GTB5"",
"clientip_status" => "missing",
"host" => {
"hostname" => "nskeylab-PowerEdge-R250"
},
"tags" => [
[0] "_grokparsefailure"
],
"event" => {
"original" => "14.49.42.25 - - [12/May/2019:01:24:44 +0000] "GET /articles/ppp-over-ssh/ HTTP/1.1" 200 "-" "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2b1) Gecko/20091014 Firefox/3.6b1 GTB5""
}
}
Why is this result？Can anyone help me?

2

You are getting a _grokparsefailure because %{NUMBER:bytes} is a required field in your pattern, but, as you know, it is missing in the second log line. So make it optional. Change

%{NUMBER:response} %{NUMBER:bytes} "(?:%{URI:referrer}|-)"

to

%{NUMBER:response} (%{NUMBER:bytes} )?"(?:%{URI:referrer}|-)"
3

Thank you very much, I'm just starting to learn these contents and I'm a bit unfamiliar with Grok。So if I want to determine if every attribute is missing, I need to process all attributes in this way?