APM Time Complexity Questions

[LordMathis (Matt Irwin)]

Kibana version: 7.3

Elasticsearch version: 7.3

APM Server version: 7.3

APM Agent language and version: C#

Hey APM community,

Are there some more exact metrics on how much time complexity APM adds to a particular program?

I ran some pretty rudimentary tests on my machine and was wondering if there are some more official numbers. My basic test was just running the same function, one with APM off:

And one with it on:

The function itself is really just designed to take up some time.

The number differences were this:

With it off:
Elapsed Time: 5.5873448 seconds
Elapsed Time: 5.7728854 seconds
Elapsed Time: 5.7654855 seconds
Elapsed Time: 5.7190262 seconds
Elapsed Time: 5.9011684 seconds

With it on:
Elapsed Time: 13.7355453 seconds
Elapsed Time: 12.0428342 seconds
Elapsed Time: 13.19944 seconds
Elapsed Time: 12.2065772 seconds
Elapsed Time: 13.244451 seconds

Which seems like a relatively steep increase. Obviously this isn't perfectly representative of the actual code we'd be running but it would be a lot more complicated to write code that would be closer to the real code, so I was hoping the APM community might have more metrics on this available.

[Sergey_Kleyman]

Hi @LordMathis,

Unfortunately we still didn't get to measuring .NET Agent performance overhead for the representative use cases - we have an issue for this task in the backlog and I hope we will get to it soon.

Regarding your particular example - I would still advise measuring a use case that is closer to your actual application code - at least this is what we plan to do for the planned benchmarks.

[karlpokus]

@LordMathis Do you really think your benchmark is fair? In the second iteration (with the apm agent enabled) you do lots of system calls to stdout. I count 4 of them before you call Watch.Stop(). I/O is not cheap and would likely increase the elapsed time you posted.

[LordMathis (Matt Irwin)]

This is a good point. Didn't think of this!

[karlpokus]

@LordMathis No worries. May I suggest you re-run your tests and update this thread with the result? I'm curious about the (actual) overhead too