Using APM Server Intake API from PHP

Elastic Observability APM
1

Hi!

I am trying to use the APM Server Intake API to monitor a high loaded PHP service which runs on many machines
It comes out, that a POST request to APM Server takes from 18ms to 80ms in average, which increases the PHP app response time significantly

Is there any method to send spans data to APM Server via UDP for example?

The last idea I came up with is to make an intermediary service which will listen an UDP port, receive and buffer APM data, and then just proxy it to APM Server via it's standard Intake API

But if there is an opportunity to send APM data via UDP by standard Elastic means it would be great! (maybe use Beats with UDP input like a proxy)

Thanks a lot!

2

Hi Andrew and thanks for trying out Elastic APM!

The requests to the APM Server are supposed to be executed by a background thread so that it does not slow down the response time of your application.

There are also a few community PHP agents on GitHub if you need some inspiration.

Cheers,
Felix

3

Felix, thanks for the reply

It is a language specific problem with the absence of so-called background threads, and community PHP agents dont solve it :frowning:

Maybe I should try to execute requests after returning response to the client

4

Hello, sorry to hear - that is indeed unfortunate.

Maybe this works well enough for you: you can run your APM Server(s) in the same machines as your instrumented service(s) and configure it so to communicate through a Unix Domain Socket: https://github.com/elastic/apm-server/blob/a9b4da1f7f77cdfbc572278550dd482daf0b3554/apm-server.yml#L6

Doing so, all the inter-process communication is handled by the OS and you save all the network cost.

Other than that, you can ensure that the APM Server is not saturated (ie not rejecting requests with a 5XX error code), by sampling requests if you have too many, trying to send more events at once but less frequently, etc.

I hope this can improve latency to an acceptable range for you.

If you try it out, please let us know how it goes!

Juan

5

Hello! Thanks for your reply

Actually, I used exactly the same method that you described
But there was a small issue, if APM server queue is saturated, it keeps the connection from agent (PHP) in "connecting" state for 1 second, to allow golang scheduler to do some work (which is too much for us), you can check this PR: https://github.com/elastic/apm-server/pull/156

We had to place NGINX in front of APM server to handle connection timeouts

But now it works like a charm

6

Glad to hear you got it working!
I'm taking note of your approach as input for future developments with out queue handling.

Have a good day

7

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.