Logstsh parce grok add teg _grokparsefailure

[bsilsy (sergey)]

good day everyone!
I'm trying to parse field "message" with grock but, i get _grokparsefailure tag on 100% of documents

send data using filebeat

message log example

"message": "10.10.10.10 - text [15/Feb/2022:15:27:27 +0300] 0.170 \"POST /text/ HTTP/1.1\" 200 735 \"-\" \"-\" \"text\" \"-\" \"-\" \"text\" \"111111111111\" \"text\""

my conf pipeline

input {
  beats {
        host => "0.0.0.0"
        port => 5044
  }
}

filter {
                      grok {
        match => { "message" => "%{IPV4:ip} - %{WORD:text1} \[%{HTTPDATE:timestamp}\] %{NUMBER:duration} \\"%{WORD:text2} \/%{WORD:text3}/ HTTP/%{NUMBER:httpversion} *?%{DATA:rawrequest} %{NUMBER:response} %{NUMBER:bytes} \\"-\\" \\"-\\" \\"%{WORD:text4}\\" \\"-\\" \\"-\\" \\"%{WORD:text5}\\" \\"%{NUMBER:id}\\" \\"%{WORD:text5}\\"" }

      }
    }

output {
        elasticsearch {
            hosts    => "localhost:9200"
            index    => "myindex-%{+YYYY.MM.dd}"
            
        }
}

it is working at grok debugger

i need any help, thanks!

[Badger]

Try

grok { match => { "message" => '%{IPV4:ip} - %{WORD:text1} \[%{HTTPDATE:timestamp}\] %{NUMBER:duration} "%{WORD:text2} \/%{WORD:text3}/ HTTP/%{NUMBER:httpversion}" *?%{DATA:rawrequest} %{NUMBER:response} %{NUMBER:bytes} "-" "-" "%{WORD:text4}" "-" "-" "%{WORD:text5}" "%{NUMBER:id}" "%{WORD:text5}"' } }

Note that you have two [text5] fields.

[bsilsy (sergey)]

@Badger

Note that you have two [text5] fields.

I renamed one of the Text5 fields

I tried changing the " to ' as you indicated, but it didn't help. I also get get _grokparsefailure tag on 100% of documents.

if I add to the filter, if [event][module] == "nginx", then I get 50-60% _grokparsefailure tag, but the fields I need are not visible.

I apologize for not the best screenshots

in the log I get

[logstash.outputs.elasticsearch] Using a default mapping template {:es_version=>7, :ecs_compatibility=>:disabled}

tried adding a string to output, but it didn't change anything

ecs_compatibility => "v1"

[Badger]

If your grok is not matching then see here, also here.