Spliting and increasing timestamp

Hello,
I am trying to do the following:
I have a json message with a consolidated array of values from a sensor (every x minutes).
{"id":"C2","timestamp":"2020-10-20T17:07:57.260Z","gap_in_ms":20,"array_of_values":{"x":[0,-0.001,0,0.001,-0.001,0,0,0.001,-0.001,0,0.001]}}

I use the split filter to create a separate message (event) for each of the values in the array.
split { field => "[array_of_values][x]" }

So far so good. I get a message per item in the array. However, the next step is where I am lost.

For each new message, I want to increase the timestamp by the gap_in_ms amount (20ms).
(I may need to add an array length k:v as well)

i.e.

  • Original message:
    {"id":"C2","timestamp":"2020-10-20T17:07:57.260Z","gap_in_ms":20,"array_of_values":{"x":[0,-0.001,0,0.001,-0.001,0,0,0.001,-0.001,0,0.001]}}
  • message 1 becomes (original timestamp):
    {"id":"C2","timestamp":"2020-10-20T17:07:57.**260**Z","gap_in_ms":20,"array_of_values":{"x":[0,-0.001,0,0.001,-0.001,0,0,0.001,-0.001,0,0.001]}}
  • message 2 becomes (original timestamp + gap_in_ms (20) for the 2nd element in the array):
    {"id":"C2","timestamp":"2020-10-20T17:07:57.**280**Z","gap_in_ms":20,"array_of_values":{"x":[0,-0.001,0,0.001,-0.001,0,0,0.001,-0.001,0,0.001]}}
  • message 3 becomes (second message timestamp + gap_in_ms (20) for the 3rd element in the array, or simply original timestamp + (gap_in_ms * array index)):
    {"id":"C2","timestamp":"2020-10-20T17:07:57.**300**Z","gap_in_ms":20,"array_of_values":{"x":[0,-0.001,0,0.001,-0.001,0,0,0.001,-0.001,0,0.001]}}
    ...
  • etc

I am assuming a ruby filter is the way to go for the time increase:

    ruby {
             code => 'event.set("new_timestamp", (event.get("@timestamp") + (event.get("gap_in_ms")/1000)).strftime("%Y-%m-%dT%H:%M:%S.%LZ"))'
          }

But how do I manage to put a different calculated timestamp in each new message?
Do I need to nest the ruby filter within the split filter?
I cannot find any documentation on nested filters nor examples of how could one achieve this with split.

Any help is highly appreciated. :slight_smile:

Using a ruby filter like that would work if you knew what array index the current event had come from, but split does not preserve that information.

I would suggest running a ruby filter before the split to create an array of hashes, with each hash containing a value from the array and the corresponding timestamp.

1 Like

Thank you very much. That helped me figure out a way. I used arrays instead of hashes, because I could not figure out how to add hashes inside a hash array in the form "hash_array":{ {"key:{"value",""value"}} , {"key:{"value",""value"}} , ...}. I am new to the ruby language and the methods were just giving me empty hashes.
Nevertheless, I got it working. Thanks!

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.