How to access filebeat HAProxy total response time?

[mmcnl (Matt McNeil)]

I have the exact same question as was asked in this topic, which was closed due to no responses. Namely:

Where do I find default HAProxy Timers Tr or Ta (which I assume include data/body) in Elastic Filebeat HAProxy fields?

Thank you for any input!

[mmcnl (Matt McNeil)]

@warkolm As you suggested in a PM, I opened a new topic with the same question. Is there anyone on the Elastic team who can respond? Thanks much for any input!

[leandrojmp (Leandro Pereira)]

The HAProxy module in filebeat uses this ingest pipeline to parse the log message.

The part that will parse the times is this one:

%{NUMBER:haproxy.http.request.time_wait_ms:long}/%{NUMBER:haproxy.total_waiting_time_ms:long}/%{NUMBER:haproxy.connection_wait_time_ms:long}/%{NUMBER:haproxy.http.request.time_wait_without_data_ms:long}/%{NUMBER:temp.duration:long}

Later in the pipeline the field temp.duration is renamed as event.duration using this processor:

- script:
    lang: painless
    source: ctx.event.duration = Math.round(ctx.temp.duration * params.scale)
    params:
      scale: 1000000
    if: ctx.temp?.duration != null

So, the 5 time fields from HAProxy HTTP requests, TR/Tw/Tc/Tr/Ta, will be parsed as:

haproxy.http.request.time_wait_ms
haproxy.total_waiting_time_ms
haproxy.connection_wait_time_ms
haproxy.http.request.time_wait_without_data_ms
event.duration

You then have:

• TR: haproxy.http.request.time_wait_ms
• Tw: haproxy.total_waiting_time_ms
• Tc: haproxy.connection_wait_time_ms
• Tr: haproxy.http.request.time_wait_without_data_ms
• Ta: event.duration

On a previous company I used to collect HAProxy logs, but I changed the ingest pipeline to store the fields as haproxy.TR, haproxy.Ta etc, I find that this is pretty easy for anyone who cames from a HAProxy background to understand what is the metric, the names used by elastic just added confusion to the analysts.

[mmcnl (Matt McNeil)]

Thank you very much @leandrojmp, much appreciated!!