APM rest api

Elastic Observability APM
,
1

Hey,

I'm using Kibana, ElasticSearch and APM in version 7.5.1 (fresh install).

I now want to use Application Performance Monitoring in my application, which is written in c++.
Since Elastic does not have an C++ APM Agent, I thought utilizing the REST API.

For simplicity I first want to test the REST API with Visual Studio Code with Plugin REST Client.
So currently I am sending a POST request to the endpoint http://xxx:5601/intake/v2/events (where xxx is the Elastic APM-Server) as described in https://www.elastic.co/guide/en/apm/server/current/events-api.html.
In the header I currently just have

content-type: application/json

And the body of this REST request is taken 1:1 from https://www.elastic.co/guide/en/apm/server/current/intake-api.html.

However after firing that REST request I get the following response:

HTTP/1.1 404 Not Found
kbn-name: kibana
kbn-xpack-sig: 98ef0b80a56156b387f246997c215417
content-type: application/json; charset=utf-8
cache-control: no-cache
content-length: 60
Date: Wed, 12 Feb 2020 12:48:33 GMT
Connection: close

{
  "statusCode": 404,
  "error": "Not Found",
  "message": "Not Found"
}

Where did I go wrong/what did I forget?

Thank you very much in advance.

2

Seems you are sending the request to Kibana as opposed to the APM Server

3

What would be the right endpoint?
I also tried http://vm-es7-apm:5601/intake/v2/rum/events, but I get the same response there.

4

The port of APM Server is usually 8200

5

I changed my approach.
Instead of logging via C++, I will redirect my output to C#.

I installed Elastic.Apm nuget package.
And wrote a function to test APM functionality.

Environment.SetEnvironmentVariable("ELASTIC_APM_SERVER_URLS", "http://xxx:8200");
Environment.SetEnvironmentVariable("ELASTIC_APM_SERVICE_NAME", "Your Service Name");

var transaction = Elastic.Apm.Agent.Tracer.StartTransaction("Transaction Name", ApiConstants.TypeRequest);
var span = transaction.StartSpan("Span Name", ApiConstants.TypeExternal, ApiConstants.ActionQuery);
Thread.Sleep(200);
span.End();
transaction.End();

On the ASP.NET project everything works fine with above URL and I can see logs in Kibana.
However executing above in a .NET class library there is no log in Kibana.

Intercepting the connection with Fiddler, I can see that there is an outgoing connection with the Request Header containing GET /config/v1/agents?service.name=Your+Service+Name HTTP/1.1 and the Response Header reads HTTP/1.1 403 Forbidden.

What am I doing wrong here?

6

The APM Server exposes an endpoint for APM agents to query for configuration changes. This endpoint is disabled by default, in which case you see above mentioned 403 response. It can be enabled by configuring apm-server.kibana.* accordingly in the APM Server.
APM agents do not rely on this configuration endpoint though, so this should not be the reason for agents not sending data to the APM Server.

7

Hi @spark,

as @simitt says that HTTP GET for the config can be ignored for your specific problem.

It should not matter where you put that code, it should also work with a class library, so unfortunately I don't see anything in your post that'd cause a problem here.

Maybe 1 idea: the agent does not stop the process from being terminated, so make sure the agent has enough time to send the data before the process is terminated - so if it's a console app, maybe put a thread sleep before the main method.

Otherwise - I assume this is some sample project - if you can share it with us somehow (just upload it to somewhere...) I'd be happy to take a look at it.

8

@simitt, @GregKalapos Thank you very much for your answers.

@simitt I checked the file apm-server\apm-server.yml.
There was only one entry active so far: Under node apm-server the entry host: "10.54.149.154:8200", so now I enabled under node kibana the entry enabled: true and restarted both, the service elasticsearch-apm and elasticsearch-kibana.
However with no change.

@GregKalapos What would be the preferred file hoster/storage? I created a new blank project Console App (.NET Framework) and .NET Framework 4.6.1 in Visual Studio 2017 (before it was a class library).
Then I added the library Elastic.Apm version 1.3.0 via nuget.
The only file program.cs in the project looks as follows:

using Elastic.Apm.Api;
using System;
using System.Diagnostics;
using System.Threading;

namespace ElatsticAPM
{
    class Program
    {
        static void Main(string[] args)
        {
            Thread.Sleep(500);

            ApmTest_ManualTest();

            Thread.Sleep(15000);
            Console.Read();
        }

        private static void ApmTest_ManualTest()
        {
            Thread.Sleep(500);

            Debug.WriteLine("Test ApmTest_ManualTest Start");

            string ElasticApmServerUrl = "http://10.54.149.154:8200";
            Environment.SetEnvironmentVariable("ELASTIC_APM_SERVER_URLS", ElasticApmServerUrl);
            Environment.SetEnvironmentVariable("ELASTIC_APM_SERVICE_NAME", "Your Service Name");
            Environment.SetEnvironmentVariable("ELASTIC_APM_LOG_LEVEL", "trace");

            var transaction = Elastic.Apm.Agent.Tracer.StartTransaction("Transaction Name", ApiConstants.TypeRequest);
            var span = transaction.StartSpan("Span Name", ApiConstants.TypeExternal, ApiConstants.ActionQuery);
            Thread.Sleep(200);
            span.End();
            transaction.End();

            Debug.WriteLine("Test ApmTest_ManualTest Output ends here 12574");

            Thread.Sleep(500);
        }
    }
}

and in the Console window I get the following output: https://pastebin.com/SfEkjP92

9

Did you ensure the APM Server can actually reach Kibana? You should find more details about that in the APM Server logs. Only setting apm-server.kibana.enabled: true might not be enough if any of the other default values do not fit your setup, eg the host. Please see the list of related config options.

10

Thanks for the info @spark, if you only have the 1 .cs file, then it's already enough for me, otherwise you can just put stuff on GitHub.

Regarding the sample you pasted and the logs:

From what I see in the logs, everything look ok from the agent's perspective:

[2020-02-17 11:27:22.193 +01:00][Debug] - {PayloadSenderV2} Sent items to server:
    Span{Id: 4d7c0b6f28272e93, TransactionId: bfd788c9f223a829, ParentId: bfd788c9f223a829, TraceId: e2c11b2fb42e135ca0847845a2059950, Name: Span Name, Type: external, IsSampled: True},
    Transaction{Id: bfd788c9f223a829, TraceId: e2c11b2fb42e135ca0847845a2059950, ParentId: null, Name: Transaction Name, Type: request, IsSampled: True}

This line tells that the agent successfully sent data (and specifically 1 transaction with name "Transaction Name" you capture in the sample code) to the APM Server. Are you totally sure this transaction is not captured? Is there maybe anything in the server logs?

11

@simitt Your hint helped to the solution.
After editing node kibana, entry host to the IP (in my case 10.54.149.154:5601), I was able to create a connection.

Thank you very much for helping me!

Now there is only one detail question: In the code you can see that I set environmental variables like ELASTIC_APM_SERVER_URLS and ELASTIC_APM_SERVICE_NAME.
In an ASP.NET project I would set these values in an AppsSettings.config file.
Is is possible to set these variables on another way in an console/class library project?

12

This depends on how the agent will be started - if the class library is in an ASP.NET Classic application and you use the IIS module, then it'll pick up the configs from web.config as described here - same for ASP.NET Core and appsettings.json.

But if you let's say embed the class library into a .NET Framework console application, then in that case the agent will only read environment variables - even if you manually parse an ApSettings.config file - there would be a way to pass the config to the agent - we can discuss that, but that'll need some coding on your side.

I guess the ASP.NET + web.config will cover your scenario, but I'm not sure - if not, and using environment variables are too bad, let's follow up.

13

@GregKalapos unfortunately I have the case without the ASP.NET application.
I only have a .NET Framework class library that will be called via RPC by c++ code of my application.

How would the coding you proposed look like?

14

Oh, I see. Ok, so in this case the only way to configure the agent is to use environment variables. All you need to do is to make sure that the environment variables are set and visible to the process that hosts the .NET Agent - like on windows just use set ELASTIC_APM_SERVER_URLS=yourserver and so on. By default the agent will always try to read configs from environment variables.

15

Well, what I said above isn't really the full story - you can still read configs through Appsettings.config- I added a sample here.

using Elastic.Apm;
using Elastic.Apm.Config;
using Elastic.Apm.Helpers;
using Elastic.Apm.Logging;
using System;
using System.Collections.Generic;
using System.Configuration;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace ConfigSample
{
	class Program
	{
		static void Main(string[] args)
		{
			Agent.Setup(new AgentComponents(configurationReader: new CustomConfig()));

			//after this the agent will use the configs from your CustomConfig
		}
	}


	class CustomConfig : IConfigurationReader
	{

		public IReadOnlyList<Uri> ServerUrls
		{
			get
			{
				var serverUrl = ConfigurationManager.AppSettings["ServerUrls"];
				var serverUri = new Uri(serverUrl);
				return new List<Uri> { serverUri };
			}

		}

		//implement all other properties
	}
}

App.Config:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
    <startup> 
        <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.7.2" />
    </startup>
	<appSettings>
		<add key="ServerUrls" value="http://localhost:8200" />
	</appSettings>
</configuration>

You'll also need to reference the System.Configuration.dll in order to use ConfigurationManager.

We are not very happy with the IConfigurationReader because you need to implement all the settings that the agent can read (A help for this could be to look at the docs about the config settings), so, yeah, this is something we think about improving somehow.

Nevertheless I'd say setting the environment variables for the given process is in general in my opinion the easiest.

16

Thank you very much, I'll give it a try.

17

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.