Non-http request for APM server

Elastic Observability APM
1

Sorry for deleting the template but this is more of is there a way than a specific issue.
I'm trying to set up a custom APM pipeline and my initial thoughts were that I can install the agents in various containers, have one APM server on a different node which will forward the data to Elasticsearch. But the agents have a big performance impact if I use an HTTP connection (not to mention other issues with this) so I thought I could simply write out the events and ship those to the APM server. Is it possible to set up APM server to receive the events which are shipped from a file?

2

Currently you can only send data to the APM Server using HTTP. In theory you could write to a file and then use curl or a similar tool to send the data in a separate step, but this isn't something we directly support today.

But the agents have a big performance impact if I use an HTTP connection (not to mention other issues with this)

What kind of performance impact are you seeing? Have you tried writing the events to disk? How does it compare in terms of performance, given you have introduced disk I/O overhead? What are the other issues you have with using HTTP?

3

The performance impact affects the PHP application we use. We figured out that this is due to the way how the PHP library we are using handling HTTP request where we can't really set "fire and forget". When we wrote events to file it helped but we don't want to complicate things with another application which would translate the shipped events to HTTP request if possible. Since we ship a lot of various event logs from the containers (from application logs) to a centralized processing cluster we want to have the same for APM.

1 Like
4

Thanks for the additional context, now I see where you're coming from. This is not currently on our roadmap, but we'll keep it in mind. Doing it yourself is possible by translating from a file to HTTP, but probably would complicate things a fair bit as you mentioned.

5

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.