Measuring multiple event durations with elapsed plugin

[joconner (John O'Conner)]

My event stream log items look like this:
<timestamp>, <event_id>, <resource_id>, <resource_type>

My event_id fields can be several strings that look like this:

• FOO_BEGIN
• FOO_END
• BAR_BEGIN
• BAR_END
• BUZZ_BEGIN
• BUZZ_END

For any particular resource_id, I have multiple BEGIN/END pairs for various workflow events on a resource.

Since the elapsed plugin uses tags, not events, I also have a mutate filter that adds the FOO_BEGIN/FOO_END, BAR_BEGIN/BAR_END event names as tags to those logged items. I'm planning to use these tag names in the elapsed blocks and use the the literal string "resource_id" in the unique_id_field like this:

elapsed {
    start_tag => "FOO_BEGIN"
    end_tag => "FOO_END"
    unique_id_field => "resource_id"
    timeout => 600
}

My questions:

1. If I want to measure the elapsed time between several BEGIN/END pairs, do I have to create a separate elapsed logstash item for each? That is, do I need a separateelapsed structure for each event pair: FOO, BAR, BUZZ? Or can I create a single elapsed block that has a variable start_tag and end_tag value? Something like this maybe:

elapsed {
    start_tag => "%{SOMEVARIABLE}_BEGIN"
    end_tag =>   "%{SOMEVARIABLE}_END"
    unique_id_field => "resource_id"
    timeout => 600
}

2. Is it ok that the elapsed blocks are outside any if block and executed even on events that are not the targets? Or do I need to put these elapsed blocks within an if structure to isolate them just for those events?

Thanks for your help.

[joconner (John O'Conner)]

I tried using a single elapsed structure with variables in the start_tag and end_tag fields. This failed miserably. Since I need to track ~20 different begin/end pairs, I've had to create 20 specific elapsed blocks to handle each one separately. The variable syntax within an elapsed block doesn't appear to work as planned.