How to access filebeat HAProxy total response time?

Elastic Stack Beats
1

I have the exact same question as was asked in this topic, which was closed due to no responses. Namely:

Where do I find default HAProxy Timers Tr or Ta (which I assume include data/body) in Elastic Filebeat HAProxy fields?

Thank you for any input!

2

@warkolm As you suggested in a PM, I opened a new topic with the same question. Is there anyone on the Elastic team who can respond? Thanks much for any input!

3

The HAProxy module in filebeat uses this ingest pipeline to parse the log message.

The part that will parse the times is this one:

%{NUMBER:haproxy.http.request.time_wait_ms:long}/%{NUMBER:haproxy.total_waiting_time_ms:long}/%{NUMBER:haproxy.connection_wait_time_ms:long}/%{NUMBER:haproxy.http.request.time_wait_without_data_ms:long}/%{NUMBER:temp.duration:long}

Later in the pipeline the field temp.duration is renamed as event.duration using this processor:

- script:
    lang: painless
    source: ctx.event.duration = Math.round(ctx.temp.duration * params.scale)
    params:
      scale: 1000000
    if: ctx.temp?.duration != null

So, the 5 time fields from HAProxy HTTP requests, TR/Tw/Tc/Tr/Ta, will be parsed as:

haproxy.http.request.time_wait_ms
haproxy.total_waiting_time_ms
haproxy.connection_wait_time_ms
haproxy.http.request.time_wait_without_data_ms
event.duration

You then have:

  • TR: haproxy.http.request.time_wait_ms
  • Tw: haproxy.total_waiting_time_ms
  • Tc: haproxy.connection_wait_time_ms
  • Tr: haproxy.http.request.time_wait_without_data_ms
  • Ta: event.duration

On a previous company I used to collect HAProxy logs, but I changed the ingest pipeline to store the fields as haproxy.TR, haproxy.Ta etc, I find that this is pretty easy for anyone who cames from a HAProxy background to understand what is the metric, the names used by elastic just added confusion to the analysts.

1 Like
4

Thank you very much @leandrojmp, much appreciated!! :slight_smile:

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.