Groke parse failure on Haproxy logs while debugger is OK

Hello,

I have a weird situation where the data is correctly parsed, grok debugger doesn't return errors. But logstash still adds a grokeparse_failure tag.

Here is an exemple log (in /var/log/haproxy.log) :

Mar  1 15:40:24 example-host-01 haproxy[1944764]: 200.100.200.100:42378 [01/Mar/2023:15:40:24.589] www.example.com example/example-host-01 0/0/0/1/1 404 415 - - ---- 27/27/8/8/0 0/0 {12.34.45.56} "GET /serviceworker.js?v=1677679200 HTTP/1.1"

My logstash grok configuration :

    if "haproxy" in [tags] {
        grok {
####         match => { "message" => "%{SYSLOGTIMESTAMP:timestamp} %{DATA:aloha_name} haproxy\[%{DATA:haproxy_process}\]: %{DATA:client_ip}:%{DATA:client_port} \[%{DATA:accept_date}\] %{DATA:frontend_name} %{DATA:backend_name}/%{DATA:server_name} %{DATA:Tq}/%{DATA:Tw}/%{DATA:Tc}/%{DATA:Tr}/%{DATA:Tt} %{DATA:http_status_code} %{DATA:bytes_read} %{DATA:request_cookie} %{DATA:response_cookie} --%{DATA:termination_state} %{DATA:actconn}/%{DATA:feconn}/%{DATA:beconn}/%{DATA:srvconn}/%{DATA:retries} %{DATA:srv_queue}/%{DATA:backend_queue} {%{DATA:remote_ip_original}} \"%{DATA:method} %{DATA:request} HTTP/%{DATA:http_version}\""}
            match => { "message" => "%{DATA:all-message}$"}
            remove_field => [ "message" ]
        }
        date {

            match        => [ "timestamp" , "MMM dd HH:mm:ss", "MMM  d HH:mm:ss"]
            target       => "@timestamp"
            remove_field => "timestamp"
        }
    }

I commented the original filter but every field is correctly provided and the same behaviour is observed

When I log the groke parse failures on logstash service, I can see this logs :

{
  "@version": "1",
  "input": {
    "type": "log"
  },
  "host": {
    "name": "example-host-01"
  },
  "agent": {
    "ephemeral_id": "13f92298-1b05-49a2-a3b9-75564cba9cff",
    "type": "filebeat",
    "version": "8.5.2",
    "id": "79d545fb-637c-45c8-88f8-0cb6a0634195",
    "name": "example-host-01"
  },
  "event": {
    "original": "Mar  1 16:37:02 example-host-01 haproxy[1944764]: 100.200.200.100:41388 [01/Mar/2023:16:37:02.493] www.example.com example/example-host-01 0/0/0/1/1 404 413 - - ---- 57/57/2/2/0 0/0 {12.34.45.56} \"GET /serviceworker.js?v=1677682800 HTTP/1.1\""
  },
  "ecs": {
    "version": "8.0.0"
  },
  "log": {
    "offset": 238383290,
    "file": {
      "path": "/var/log/haproxy.log"
    }
  },
  "tags": "example-haproxy-_grokparsefailure",
  "@timestamp": "2023-03-01T15:37:02.630Z",
  "all-message": "Mar  1 16:37:02 example-host-01 haproxy[1944764]: 100.200.100.200:41388 [01/Mar/2023:16:37:02.493] www.example.com example/example-host-01 0/0/0/1/1 404 413 - - ---- 57/57/2/2/0 0/0 {12.34.45.56} \"GET /serviceworker.js?v=1677682800 HTTP/1.1\""
}

On elastic, I can see all fields well provided, so everything is working normally except that the indices have that _grokparsefailure tag added. Every log has this behaviour

Am I doing something wrong ? I noticed how the logstash is adding \ elements in front of double quotes but I'm a bit clueless here.

Thank you in advance

I found out, one of my tags was matching with another filter in my logstash configuration.

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.