Measuring multiple event durations with elapsed plugin

My event stream log items look like this:
<timestamp>, <event_id>, <resource_id>, <resource_type>

My event_id fields can be several strings that look like this:


For any particular resource_id, I have multiple BEGIN/END pairs for various workflow events on a resource.

Since the elapsed plugin uses tags, not events, I also have a mutate filter that adds the FOO_BEGIN/FOO_END, BAR_BEGIN/BAR_END event names as tags to those logged items. I'm planning to use these tag names in the elapsed blocks and use the the literal string "resource_id" in the unique_id_field like this:

elapsed {
    start_tag => "FOO_BEGIN"
    end_tag => "FOO_END"
    unique_id_field => "resource_id"
    timeout => 600

My questions:

  1. If I want to measure the elapsed time between several BEGIN/END pairs, do I have to create a separate elapsed logstash item for each? That is, do I need a separateelapsed structure for each event pair: FOO, BAR, BUZZ? Or can I create a single elapsed block that has a variable start_tag and end_tag value? Something like this maybe:
elapsed {
    start_tag => "%{SOMEVARIABLE}_BEGIN"
    end_tag =>   "%{SOMEVARIABLE}_END"
    unique_id_field => "resource_id"
    timeout => 600
  1. Is it ok that the elapsed blocks are outside any if block and executed even on events that are not the targets? Or do I need to put these elapsed blocks within an if structure to isolate them just for those events?

Thanks for your help.

I tried using a single elapsed structure with variables in the start_tag and end_tag fields. This failed miserably. Since I need to track ~20 different begin/end pairs, I've had to create 20 specific elapsed blocks to handle each one separately. The variable syntax within an elapsed block doesn't appear to work as planned.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.