Changing the precision of the @timestamp field

[ankh]

I have been using the generated @timestamp field in our documents as a record of when a document was sent to Elastic. I have been renaming the field in Logstash to suit our document structure.

rename => { "@timestamp" => "[audit][processingTimestamp]" }

Our version of Elastic is such that the precision of the timestamp is to milliseconds, and this matches the other timestamps in the document.

Upgrading to version 8, I see that the @timestamp precision is now to the microseconds level, due to a change from a Ruby library to a Java library I think.

I want to keep the precision of this timestamp at milliseconds. What's the simplest way to do this? I could do it in 2 steps like this:

     ruby {
          code => "
              event.set('[processingMetadata][indexedDatetime]',
              event.get('@timestamp').time.strftime('%Y-%m-%dT%H:%M:%S.%LZ'))
          "
      }

      mutate {
          remove_field => [
              "@timestamp"
          ]
      }

but is there a better way? Is the format/precision of the default @timestamp configurable?

[Badger]

@ankh

I could do it in 2 steps like this

You don't need two steps. You can replace the call to event.get with event.remove.

[ankh]

Thanks for that. That's true, though I am already removing other fields from the message so the mutate filter will be there anyway. I would probably prefer to group all the removes together instead of splitting over 2 different filters.

I guess I was trying to see if there was a way of changing the precision of the timestamp without using a dedicated Ruby filter.

[Badger]

@ankh

I was trying to see if there was a way of changing the precision of the timestamp without using a dedicated Ruby filter.

I do not know of one. Perhaps you can force the field to be a date rather than a date_nanos in elasticsearch using a mapping.

[ankh]

Thanks again. The field is being mapped to the "date" datatype, which I can see is in milliseconds precision already. So queries that can be satisfied by returning the indexed data would be OK, and it's those requests that are looking at the returned source document that would run into the microseconds.

Thanks again for your input, I appreciate it.

[ankh]

The Ruby code I thought would work needs more work. Could you please help me?

      ruby { 
          code => "
              event.set('timestamp2',
              event.get('@timestamp').time.strftime('%Y-%m-%dT%H:%M:%S.%LZ'))
          "
      }

Gives this message:

{
        "@timestamp" => 2024-02-21T23:26:22.717880087Z,
        "timestamp2" => "2024-02-22T10:26:22.717Z",
}

So the timestamp2 is now at milliseconds, which is good, but also has changed timezone but I want to keep it in UTC. It's probably very simple, but how can I keep timestamp2 in UTC?

[Badger]

    ruby {
        code => '
            t = event.get("@timestamp").time
            event.set("timestamp2", (t - t.utc_offset).strftime("%Y-%m-%dT%H:%M:%S.%LZ"))
        '
    }

The StackOverflow question I got that from says it should be t + t.utc_offset, but that didn't work for me.

[ankh]

Thanks again, I appreciate it. That works!

I tried to step through that to see where the time shifts are happening, and it looks like it's the strftime:

Config:

      ruby {
          code => "
              t = event.get('@timestamp').time
              t_utc_offset = t.utc_offset
              t_minus_t_utc_offset = t - t.utc_offset

              t_minus_t_utc_offset_formatted = t_minus_t_utc_offset.strftime('%Y-%m-%dT%H:%M:%S.%LZ')

              event.set('t', t)
              event.set('t_utc_offset', t_utc_offset)
              event.set('t_minus_t_utc_offset', t_minus_t_utc_offset)
              event.set('t_minus_t_utc_offset_formatted', t_minus_t_utc_offset_formatted)
          "
      }

      mutate {
          add_field => { "timestamp_mutate1" => "%{+yyyy-MM-dd'T'HH:mm:ss.SSSZ}" }
          add_field => { "timestamp_mutate2" => "%{+yyyy-MM-dd'T'HH:mm:ss.SSS'Z'}" }
      }

Output:

{
                    "@timestamp" => 2024-02-22T05:04:59.702136992Z,
                             "t" => 2024-02-22T05:04:59.702136992Z,
                  "t_utc_offset" => 39600,
          "t_minus_t_utc_offset" => 2024-02-21T18:04:59.702136992Z,
"t_minus_t_utc_offset_formatted" => "2024-02-22T05:04:59.702Z",
             "timestamp_mutate1" => "2024-02-22T05:04:59.702+0000",
             "timestamp_mutate2" => "2024-02-22T05:04:59.702Z",
}

So

t = @timestamp = current time in UTC
t.utc_offset = 11 hours
(t - t.utc_offset) = current time in UTC - 11 hours (some complete other timezone)
(t - t.utc_offset).strftime = current time in UTC, formatted correctly

So it seems to work by backshifting the time by the tz difference, to negate the later tz shift forward. It seems odd that strftime would do that time shifting?

I also tried doing it using mutate to create new fields without using timestamp, and it sort of worked, but to get the "Z" tz indicator I had to hard code it, although I guess there's another way to do that. But I just realised the date format I used in the Ruby filter also hard codes the Z I think...

Thanks again!

[Badger]

@ankh

It seems odd that strftime would do that time shifting?

Yeah, sort of. But in the elastic stack we live in a world where everything has to be UTC. That's not where most folks live. I can definitely see why some people would think time-shifting in the library is useful.