Calculate the time difference between 2 xml log lines

Elastic Stack Logstash
1

Hello everyone,
I have an XML log file that looks like this:

<TRACE timestamp="1612778642004" dateTimeFormat="yyyy.MM.dd kk:mm:ss z">
    <LOGIN user="BSCSWS" factory="117" timestamp="1612778642003"/>
    <SOCREATE soiName="CMI" soiVersion="2" name="144" timestamp="1612778642005" factory="117"/>

    <COMMAND name="SESSION.CHANGE" timestamp="1612778642014" so="144">
...
    </COMMAND>
    <COMMAND name="LANGUAGES.READ" timestamp="1612778642017" so="144">
...
    </COMMAND>
<SODISPOSE soiName="CMI" soiVersion="2" name="144" timestamp="1612779491299" factory="117"/>
</TRACE>

I would like to get the execution time of each command, that means I should calculate the difference between two successive timestamps. I tried to use the elapsed plugin, but nothing is displayed in output.
Is there any other way to do this?

Thank you for your help :slight_smile:

2

How are you consuming the log? Is it a line at a time or are you consuming the entire TRACE element as a single event?

3

I'm consuming each COMMAND element as a single event.

4

Assuming that each event has a [timestamp] field with a value like "1612778642014" I would do this in a ruby filter. You will need to set pipeline.workers to 1 and make sure pipeline.ordered is true (which it is by default in 7.x when pipeline.workers is 1)

ruby {
    code => '
        ts = event.get("timestamp").to_i
        if ts
            if @lastTimestamp
                event.set("duration", ts - @lastTimestamp)
            end
            @lastTimestamp = ts
        end
    '
}
5

This worked perfectly, thank you :slight_smile: