"timestamp.us" field in Elastic APM Delay

Elastic Observability APM
1

Hi ,

I am Using Elasticsearch version 8.6.2 with Elastic APM java Agent.

While parallelly running 3 to 4 scenarios of my application, it seems the stop-time time of my application does not match the timestamp.us of the transaction traces. Due to this, Its affecting the application's business logic.

Example: the stop-time of my scenario captured was 12:20 PM and the timestamp.us is 12:21 PM when converted into timestamp

The stopTime which is System.currentTimeMillis() in my application logic is lesser than the timestamp.us field in the transaction traces.

This Stop-time is captured after the execution of the entire scenario.

So, My Assumption is that timestamp.us should be lesser than or equal to the stop-time of my application's scenario.

Can someone please help me understand what time does timestamp.us exactly denote? Is it the transaction response time or the time the transaction was logged in elasticsearch?

In the current issue that I am facing in my application, I assume that there might be a lag in the transaction traces been sent to the apm-server.

Your advise/ Suggestion would be of great help.

Thanks & Regards,
Namita Jaokar

2

timestamp.us on spans and transactions is recorded by the agents, therefore the time is based on the clock of your monitored system. The timestamp is in microseconds since the UNIX epoche, while System.currentTimeMillis() is in milliseconds.

3

Hi @Jonas_Kunz ,

By the agent you mean the Elastic APM Agent jar , correct?
I have the logic added wherein the microseconds to milliseconds conversion is handled.

What I understand here is that timestamp.us gives the time of the system-(wherein the application under observability is deployed ) that is configured with the agent.
What is the difference between @timestamp and timestamp.us apart from the timestamp format ? Is there any other major differences between the two?

Thanks & Regards,
Namita Jaokar

4

By the agent you mean the Elastic APM Agent jar , correct?

Correct.

What is the difference between @timestamp and timestamp.us apart from the timestamp format ? Is there any other major differences between the two?

There should be no difference apart from the format. I think timestamp.us was introduced because @timestamp only has millisecond precision, which in many cases is not enough.