Create Transactions from logs in .net

Elastic Observability APM
1

Hello,

I have a use case where we have logs generated in Azure blob from our deployments. Now we need to implement tracing for these requests. Since the architecture here is different we cannot directly add APM Library call to create spans and transactions in our deployed code. So what we need to do is read log files and use them to create traces and span. However I am not sure if this is possible as it wont be able to capture exact timing informations and necessary tags and labels.

Please guide on what can be done to trace this scenario or if it is even possible to do so.

2

Hi @faizal,

the .NET Agent was not really planned for this use-case, so as you correctly say, this is something which is not really possible with it it at the moment.

There is an issue which is about to implement something that'd support your scenario, but we currently don't actively work on that - but it's definitely worth watching in your case.

So to be honest there is no out-of-the-box library you could use currently - or at least I'm not aware of that.

One option you have is to implement something that reads the logs and sends it to the APM Server. Here is the documentation of the intake API of the Elastic APM Server. So basically you need to turn your logs into the format defined in the docs and with that you could send those too APM Server. Unfortunately this'd require some coding.

3

Hi @GregKalapos thanks I will have a look at the intake API's and write some code for that.

If that works out will it work similar to how the normal agent works I mean will we get all the analytics , like how we get for while using the normal .net agent?

Also are there any examples available for use of these intake API's?

4

If that works out will it work similar to how the normal agent works I mean will we get all the analytics , like how we get for while using the normal .net agent?

Well, not 100% - or I should say: it depends on what you implement. Based on your description I assumed it'd be enough to just implement the event API - like sending transactions, spans, and errors - so you will see all those events. But for example you won't see metrics and of course all the auto instrumentation features will be missing, too. One option you can do here is that you add the agent to the process (not sure this is possible in your case), but you configure it to not capture any transaction - with this the normal agent can collect metrics, and your library can "recreate" transactions, spans and errors from your logs.

Also are there any examples available for use of these intake API's?

I think the docs I linked are the best to look at - when I started working on the .NET Agent I mostly went based on the APM Server docs. Other useful resource is here on the Server repo the spec of the API. You can also look at the .NET Agent source code. In this folder you can find the data transfer objects we send to the APM Server and this class sends the data.

5

Thank you , I will have a look.

Just one more query we are also using .net Bot Framework for creating Chatbot, can we use the Agent for tracing here. I know there is no out of the box support for this according to docs but can the public Agent API be used here to manually create transactions and spans where ever needed.

6

@faizal Correct, you can use the public API in that scenario - basically all you need is either .NET Framework 4.6.1 or newer or any .NET Flavor that supports .NET Standard 2.0 and the process needs to be able to send HTTP requests to the APM Server.

One thing that could be an issue here is that the agent won't hold up the process when it terminates - I'm not very familiar with the Bot Framework, but if the process terminates immediately, the agent won't be able to send the data to the APM Server - We know this is a problem in many scenarios, so we already have an issue for this. One (not so nice) solution would be to just hold up the process for a couple of seconds before it terminates and set the FlushInterval to something low (like 1 sec).

In case the way it works is that the process is not terminated then you can ignore this.

7

Hello @GregKalapos Hope you are fine!.

I was trying the events Intake API example from https://www.elastic.co/guide/en/apm/server/current/example-intake-events.html

for testing purpose I am trying this using Postman first. I copied the same data from the example and used this Post API:

http://localhost:8200/intake/v2/events

but it gives error that the content type is invalid for both "application/json" and "text/plain". what is the required content type here.

8

Hi @faizal,

It's ndjson: " Each event is sent as its own line in the HTTP request body. This is known as newline delimited JSON (NDJSON)." See here. It should be application/x-ndjson. Here is how the .NET Agent does it.

9

Hello @GregKalapos

Thanks a lot for your help I was able to implement tracing using combination of public agent APIs and events APIs.

Also just to know, when is the opentelemetry implementation (or inclusion), that you mentioned earlier, planned for? as that seems to be a proper solution and will make our app loosely coupled with Elastic APM in long run.

I know you mentioned that it is not on priority right now, So can we implement the Exporter for sending traces to elastic APM using the events intake API's ourselves?

10

Hi @faizal,

Also just to know, when is the opentelemetry implementation (or inclusion), that you mentioned earlier, planned for?

Currently we don't actively work on that, mainly because it's in a super early stage (latest version is 0.2.0-alpha). So this really depends on how the .NET implementation of opentelemetry evolves.

So can we implement the Exporter for sending traces to elastic APM using the events intake API's ourselves?

This is definitely a valid approach and - to be very open here - short term probably there won't be an out of the box solution for your use-case, so if you have enough time for this then I'd go for it.

11

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.