[HAproxy] log pipeline fails to extract http.request.method for HTTP/2.0 requests

Elastic Stack Beats
1

Greetings,

in accordance with your Github guideline, I am reporting this issue here.

Please post all questions and issues on Beats - Discuss the Elastic Stack
before opening a Github Issue. Your questions will reach a wider audience there,
and if we confirm that there is a bug, then you can open a new issue.

The HAproxy Log parser implementation fails to parse the http.request.method with HAproxy log lines generated by HTTP/2.0 requests. The reason behind that is, that HAproxy logs the full URL for requests made with HTTP/2.0 as opposed to only the path as it was in HTTP/1.1.

The grok expression can be found here: https://github.com/elastic/beats/blob/master/filebeat/module/haproxy/log/ingest/pipeline.yml#L63

%{WORD:http.request.method}%{SPACE}%{URIPATHPARAM:url.original}%{SPACE}HTTP/%{NUMBER:http.version}

Executing that expression against a HTTP/1.1 logline works as expected:
Line

POST /credentials HTTP/1.1

Result:

{
  "http": {
    "request": {
      "method": "POST"
    },
    "version": "1.1"
  },
  "url": {
    "original": "/credentials"
  }
}

Executing that expression against a HTTP/2.0 logline delivers wrong results:
Line:

GET https://subdomain.domain.tld/test HTTP/2.0

Result:

{
  "http": {
    "request": {
      "method": "tld"
    },
    "version": "2.0"
  },
  "url": {
    "original": "/test"
  }
}

(Note the method was extracted as tld)

Relevant HAproxy issue: Log-format %HU for http/2 requests logs full url (including protocol) - Help! - HAProxy community

Note: This seems to be an intended change for HTTP/2.0

The implementation for the parsing http.request.method was originally discussed here:

Best Regards
Mydayyy

Note:

I fixed it by adding another pattern for the full url before the pattern from above:

      "grok": {
        "field": "haproxy.http.request.raw_request_line",
        "ignore_missing": true,
        "patterns": [
          "%{WORD:http.request.method}%{SPACE}%{URI:url.original}%{SPACE}HTTP/%{NUMBER:http.version}",
          "%{WORD:http.request.method}%{SPACE}%{URIPATHPARAM:url.original}%{SPACE}HTTP/%{NUMBER:http.version}"
        ]
      }
    },
2

Hey @Mydayyy, welcome to discuss :slight_smile: and thanks a lot for reporting this problem! I have added it to this issue with a list of problematic logs with current haproxy integration: Increase support of log formats in haproxy filebeat module · Issue #3250 · elastic/integrations · GitHub

1 Like
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.