We already have an inhouse developed APM tool which works on the same lines as aws X-Ray works. It has same structure of transaction, span etc.
There are many cases for us where the log size of APM can go beyond 4 MB (it contains 3rd part http request and response which we need to save). We have created a logic where we can still relay all these logs with least latency (under 70 ms) and save data in aws S# aswell.
Our current stack uses Redis, Mongodb, Inhouse developed tool, SQS. An its very reliable.
Now we want to take advantage of Elasticsearch APM as well as out of box dashboards and metrics.
My question is that can we bypass the need of APM server to ingest logs from the .net agent and push those to elastic search directly? I understand we will have to modify our log structure to what Elasticsearch can understand.