# Help with grok pattern for haproxy **URL:** \ **Category:** Logstash\ **Created:** [January 17, 2020, 10:51am UTC](https://discuss.elastic.co/t/help-with-grok-pattern-for-haproxy/215441 "2020-01-17T10:51:14Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![mmikulic](https://avatars.discourse-cdn.com/v4/letter/m/8797f3/32.png) [@mmikulic](https://discuss.elastic.co/u/mmikulic)\ **Post date:** [January 17, 2020, 10:51am UTC](https://discuss.elastic.co/t/help-with-grok-pattern-for-haproxy/215441/1 "2020-01-17T10:51:14Z")
Hi,please assit with creating custom grok pattern for logstash. I've created a custom log format in haproxy conf for purpose of logging mutual ssl authentication details for client certificates. `log-format %ci:%cp\ [%t]\ %ft\ %b/%s\ %TR/%Tw/%Tc/%Tr/%Ta\ %ST\ %B\ %CC\ %CS\ %tsc\ %ac/%fc/%bc/%sc/%rc\ %sq/%bq\ %hr\ %hs\ {%[ssl_c_verify],%{+Q}[ssl_c_s_dn],%{+Q}[ssl_c_i_dn]}\ %{+Q}r\ %sslv\ %sslc` Log line example looks like this: `Jan 17 08:47:06 localhost haproxy[9870]: 10.251.121.117:10300 [17/Jan/2020:08:47:06.531] api.example.com~ test-backend/server1 0/0/1/8/9 201 589 - - ---- 421/419/1/1/0 0/0 {||10.105.74.60} {331260} {0,"/C=GB/L=Tester/O=Some Company That Connects Co./CN=api.somecompany.com","/C=US/O=CertAuth, Inc./OU=See www.certauth.net/legal-terms/OU=(c) 2012 CertAuth, Inc. - for authorized use only/CN=CertAuth Certification Authority - K1B"} "POST /some/1/path HTTP/1.1" TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256` When using standard haproxy grok pattern with additions for tls version and cipher: `%{SYSLOGTIMESTAMP:syslog_timestamp} %{IPORHOST:syslog_server} %{SYSLOGPROG}: %{IP:client_ip}:%{INT:client_port} \[%{HAPROXYDATE:accept_date}\] %{NOTSPACE:frontend_name} %{NOTSPACE:backend_name}/%{NOTSPACE:server_name} %{INT:time_request}/%{INT:time_queue}/%{INT:time_backend_connect}/%{INT:time_backend_response}/%{NOTSPACE:time_duration} %{INT:http_status_code} %{NOTSPACE:bytes_read} %{DATA:captured_request_cookie} %{DATA:captured_response_cookie} %{NOTSPACE:termination_state} %{INT:actconn}/%{INT:feconn}/%{INT:beconn}/%{INT:srvconn}/%{NOTSPACE:retries} %{INT:srv_queue}/%{INT:backend_queue} (\{%{HAPROXYCAPTUREDREQUESTHEADERS}\})?( )?(\{%{HAPROXYCAPTUREDRESPONSEHEADERS}\})?( )?"(|(%{WORD:http_verb} (%{URIPROTO:http_proto}://)?(?:%{USER:http_user}(?::[^@]*)?@)?(?:%{URIHOST:http_host})?(?:%{URIPATHPARAM:http_request})?( HTTP/%{NUMBER:http_version})?))?"( %{NOTSPACE:tls_version})?( %{NOTSPACE:tls_ciphersuite})?` Information from log line that I want to parse ends up in captured\_response\_headers: ``` "HAPROXYCAPTUREDRESPONSEHEADERS": [ [ "331260} {0,"/C=GB/L=Tester/O=Some Company That Connects Co./CN=api.somecompany.com","/C=US/O=CertAuth, Inc./OU=See www.certauth.net/legal-terms/OU=(c) 2012 CertAuth, Inc. - for authorized use only/CN=CertAuth Certification Authority - K1B"" ] ], "captured_response_headers": [ [ "331260} {0,"/C=GB/L=Tester/O=Some Company That Connects Co./CN=api.somecompany.com","/C=US/O=CertAuth, Inc./OU=See www.certauth.net/legal-terms/OU=(c) 2012 CertAuth, Inc. - for authorized use only/CN=CertAuth Certification Authority - K1B"" ] ] ``` I need to parse captured\_response\_headers so search is more easier like this: ``` "id": [ [ "331260" ] ], "ssl_verified": [ [ "0" ] ], "cert_details": [ [ "/C=GB/L=Tester/O=Some Company That Connects Co./CN=api.somecompany.com","/C=US/O=CertAuth, Inc./OU=See www.certauth.net/legal-terms/OU=(c) 2012 CertAuth, Inc. - for authorized use only/CN=CertAuth Certification Authority - K1B"" ] ] ``` ---
**Author:** ![mmikulic](https://avatars.discourse-cdn.com/v4/letter/m/8797f3/32.png) [@mmikulic](https://discuss.elastic.co/u/mmikulic)\ **Post date:** [January 17, 2020, 12:38pm UTC](https://discuss.elastic.co/t/help-with-grok-pattern-for-haproxy/215441/2 "2020-01-17T12:38:44Z")
Ok, solved it myself. For posterity that should find this thread in the future: Default `{%{HAPROXYCAPTUREDREQUESTHEADERS}\}` Should be replaced by `{%{INT:id}} {%{INT:ssl_verified},%{DATA:client_cert_details},%{DATA:cert_ca}}` The whole grok pattern should look like this: > %{SYSLOGTIMESTAMP:syslog\_timestamp} %{IPORHOST:syslog\_server} %{SYSLOGPROG}: %{IP:client\_ip}:%{INT:client\_port} [%{HAPROXYDATE:accept\_date}] %{NOTSPACE:frontend\_name} %{NOTSPACE:backend\_name}/%{NOTSPACE:server\_name} %{INT:time\_request}/%{INT:time\_queue}/%{INT:time\_backend\_connect}/%{INT:time\_backend\_response}/%{NOTSPACE:time\_duration} %{INT:http\_status\_code} %{NOTSPACE:bytes\_read} %{DATA:captured\_request\_cookie} %{DATA:captured\_response\_cookie} %{NOTSPACE:termination\_state} %{INT:actconn}/%{INT:feconn}/%{INT:beconn}/%{INT:srvconn}/%{NOTSPACE:retries} %{INT:srv\_queue}/%{INT:backend\_queue} ({%{HAPROXYCAPTUREDREQUESTHEADERS}})?( )?({%{INT:chid}} {%{INT:ssl\_verified},%{DATA:client\_cert\_details},%{DATA:cert\_ca}})?( )?"(|(%{WORD:http\_verb} (%{URIPROTO:http\_proto}://)?(?:%{USER:http\_user}(?::[^@]\*)?@)?(?:%{URIHOST:http\_host})?(?:%{URIPATHPARAM:http\_request})?( HTTP/%{NUMBER:http\_version})?))?"( %{NOTSPACE:tls\_version})?( %{NOTSPACE:tls\_ciphersuite})? ---
**Author:** ![mmikulic](https://avatars.discourse-cdn.com/v4/letter/m/8797f3/32.png) [@mmikulic](https://discuss.elastic.co/u/mmikulic)\ **Post date:** [January 17, 2020, 2:01pm UTC](https://discuss.elastic.co/t/help-with-grok-pattern-for-haproxy/215441/3 "2020-01-17T14:01:07Z")
too soon. It works on [https://grokdebug.herokuapp.com/](https://grokdebug.herokuapp.com/) But not on logstash-5.6.16-1.noarch it doesn't work. I'll try latest version ---
**Author:** ![mmikulic](https://avatars.discourse-cdn.com/v4/letter/m/8797f3/32.png) [@mmikulic](https://discuss.elastic.co/u/mmikulic)\ **Post date:** [January 22, 2020, 12:02pm UTC](https://discuss.elastic.co/t/help-with-grok-pattern-for-haproxy/215441/4 "2020-01-22T12:02:55Z")
Here is final solution for anyone who should find this `%{SYSLOGTIMESTAMP:syslog_timestamp} %{IPORHOST:syslog_server} %{SYSLOGPROG}: %{IP:client_ip}:%{INT:client_port} \[%{HAPROXYDATE:accept_date}\] %{NOTSPACE:frontend_name} %{NOTSPACE:backend_name}/%{NOTSPACE:server_name} %{INT:time_request}/%{INT:time_queue}/%{INT:time_backend_connect}/%{INT:time_backend_response}/%{NOTSPACE:time_duration} %{INT:http_status_code} %{NOTSPACE:bytes_read} %{DATA:captured_request_cookie} %{DATA:captured_response_cookie} %{NOTSPACE:termination_state} %{INT:actconn}/%{INT:feconn}/%{INT:beconn}/%{INT:srvconn}/%{NOTSPACE:retries} %{INT:srv_queue}/%{INT:backend_queue} (\{(%{DATA:request_header_x_forwarded_host})?\|(%{DATA:request_header_x_forwarded_for})?\|(%{DATA:request_header_host})?\})?\s?(\{(%{INT:id})?\})?\s?(\{(%{NOTSPACE:cert_verified})?,("%{DATA:client_cert_details}")?,("%{DATA:cert_ca}")?\}( )?")?(|(%{WORD:http_verb} (%{URIPROTO:http_proto}://)?(?:%{USER:http_user}(?::[^@]*)?@)?(?:%{URIHOST:http_host})?(?:%{URIPATHPARAM:http_request})?( HTTP/%{NUMBER:http_version})?))?"( %{NOTSPACE:tls_version})?( %{NOTSPACE:tls_ciphersuite})?` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 19, 2020, 12:02pm UTC](https://discuss.elastic.co/t/help-with-grok-pattern-for-haproxy/215441/5 "2020-02-19T12:02:59Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.