How to grok haproxy Log

Hi,

I'm parsing this log, but I get stuck in curly braces, I don't know what exception to add to export dns and ip.

This is my log:
Dec 6 10:31:43 eu01-test-test haproxy[1311]: eu01-test.test.lan x.x.x.x:xxxx [06/Dec/2022:10:31:43.545] FT_https_443~ BK_test_apache939/Ad-A 0/0/15/179/199 200 14429 - - ---- 167/166/39/39/0 0/0 {domain.com|x.x.x.x|} "POST /rvrt/tertet/ryryyr;rhrbr= HTTP/1.1"

this is my grok expression:

%{SYSLOGTIMESTAMP:timestamp} %{DATA:host_name} %{DATA:SyslogFacility} %{DATA:Haproxy_process} %{DATA:client_ip}:%{INT:client_port} [%{HAPROXYDATE:accept_date}] %{NOTSPACE:frontend_name} %{NOTSPACE:backend_name}/%{NOTSPACE:server_name} %{INT:Tq}/%{INT:Tw}/%{INT:Tc}/%{INT:Tr}/%{INT:Tt} %{INT:http_status_code} %{NOTSPACE:bytes_read} - - ---- %{INT:actconn}/%{INT:feconn}/%{INT:beconn}/%{INT:srvconn}/%{NOTSPACE:retries} %{INT:srv_queue}/%{INT:backend_queue}

can you help me?

Thanks

## These patterns were tested w/ haproxy-1.4.15

## Documentation of the haproxy log formats can be found at the following links:
## http://code.google.com/p/haproxy-docs/wiki/HTTPLogFormat
## http://code.google.com/p/haproxy-docs/wiki/TCPLogFormat

HAPROXYTIME (?!<[0-9])%{HOUR:haproxy_hour}:%{MINUTE:haproxy_minute}(?::%{SECOND:haproxy_second})(?![0-9])
HAPROXYDATE %{MONTHDAY:haproxy_monthday}/%{MONTH:haproxy_month}/%{YEAR:haproxy_year}:%{HAPROXYTIME:haproxy_time}.%{INT:haproxy_milliseconds}

# Override these default patterns to parse out what is captured in your haproxy.cfg
HAPROXYCAPTUREDREQUESTHEADERS %{DATA:captured_request_headers}
HAPROXYCAPTUREDRESPONSEHEADERS %{DATA:captured_response_headers}

# Example:
#  These haproxy config lines will add data to the logs that are captured
#  by the patterns below. Place them in your custom patterns directory to 
#  override the defaults.  
#
#  capture request header Host len 40
#  capture request header X-Forwarded-For len 50
#  capture request header Accept-Language len 50
#  capture request header Referer len 200
#  capture request header User-Agent len 200
#
#  capture response header Content-Type len 30
#  capture response header Content-Encoding len 10
#  capture response header Cache-Control len 200
#  capture response header Last-Modified len 200
# 
# HAPROXYCAPTUREDREQUESTHEADERS %{DATA:request_header_host}\|%{DATA:request_header_x_forwarded_for}\|%{DATA:request_header_accept_language}\|%{DATA:request_header_referer}\|%{DATA:request_header_user_agent}
# HAPROXYCAPTUREDRESPONSEHEADERS %{DATA:response_header_content_type}\|%{DATA:response_header_content_encoding}\|%{DATA:response_header_cache_control}\|%{DATA:response_header_last_modified}

# parse a haproxy 'httplog' line 
HAPROXYHTTP %{SYSLOGTIMESTAMP:syslog_timestamp} %{IPORHOST:syslog_server} %{SYSLOGPROG}: %{IP:client_ip}:%{INT:client_port} \[%{HAPROXYDATE:accept_date}\] %{NOTSPACE:frontend_name} %{NOTSPACE:backend_name}/%{NOTSPACE:server_name} %{INT:time_request}/%{INT:time_queue}/%{INT:time_backend_connect}/%{INT:time_backend_response}/%{NOTSPACE:time_duration} %{INT:http_status_code} %{NOTSPACE:bytes_read} %{DATA:captured_request_cookie} %{DATA:captured_response_cookie} %{NOTSPACE:termination_state} %{INT:actconn}/%{INT:feconn}/%{INT:beconn}/%{INT:srvconn}/%{NOTSPACE:retries} %{INT:srv_queue}/%{INT:backend_queue} (\{%{HAPROXYCAPTUREDREQUESTHEADERS}\})?( )?(\{%{HAPROXYCAPTUREDRESPONSEHEADERS}\})?( )?"%{WORD:http_verb} %{URIPATHPARAM:http_request}( HTTP/%{NUMBER:http_version}")?

# parse a haproxy 'tcplog' line
HAPROXYTCP %{SYSLOGTIMESTAMP:syslog_timestamp} %{IPORHOST:syslog_server} %{SYSLOGPROG}: %{IP:client_ip}:%{INT:client_port} \[%{HAPROXYDATE:accept_date}\] %{NOTSPACE:frontend_name} %{NOTSPACE:backend_name}/%{NOTSPACE:server_name} %{INT:time_queue}/%{INT:time_backend_connect}/%{NOTSPACE:time_duration} %{NOTSPACE:bytes_read} %{NOTSPACE:termination_state} %{INT:actconn}/%{INT:feconn}/%{INT:beconn}/%{INT:srvconn}/%{NOTSPACE:retries} %{INT:srv_queue}/%{INT:backend_queue}

Try if these can be of your help.

Hi dadiasish,

where did you create this config file?

Sorry if I confused you. This is not a configuration file. But it contains the grok patters for haproxy which might be useful for your reference.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.