# Troubleshooting APM Agent to Fleet Server

**URL:** <https://discuss.elastic.co/t/troubleshooting-apm-agent-to-fleet-server/318136>\
**Category:** APM\
**Tags:** dotnet\
**Created:** [November 3, 2022, 7:59pm UTC](https://discuss.elastic.co/t/troubleshooting-apm-agent-to-fleet-server/318136 "2022-11-03T19:59:50Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![sb-devora](https://avatars.discourse-cdn.com/v4/letter/s/7c8e57/32.png) [@sb-devora](https://discuss.elastic.co/u/sb-devora)\
**Post date:** [November 3, 2022, 7:59pm UTC](https://discuss.elastic.co/t/troubleshooting-apm-agent-to-fleet-server/318136/1 "2022-11-03T19:59:50Z")

</div>

Hi,  
I'm trying to set up Elastic APM on a dotnet mvc app. Testing with a dotnet core console app but my intended setup is for a regular dotnet framework project in mvc.

I have Elasticsearch and Kibana set up.  
I have my Elastic Agent running Fleet set up and self-signing ssl certs.  
Fleet status is healthy. Diagnostics look clean.

However, when I go to run the sample apm-agent-dotnet console app I get

```auto
      {CentralConfigurationFetcher} Exception was thrown while fetching configuration from APM Server and parsing it. ETag: `<null>'. URL: `https://<sb-devora-apm-server--url>:8220/config/v1/agents?service.name=SampleConsoleNetCoreApp&service.environment=Production'. Apm Server base URL: `https://<sb-devora-apm-server--url>:8220/'. WaitInterval: 5m. dbgIterationsCount: 9.
      +-> Request:
          Method: GET, RequestUri: 'https://<sb-devora-apm-server--url>:8220/config/v1/agents?service.name=SampleConsoleNetCoreApp&service.environment=Production', Version: 1.1, Content: <null>, Headers:
          {
            User-Agent: apm-agent-dotnet/1.18.0
            User-Agent: (SampleConsoleNetCoreApp 1.0.0)
            User-Agent: System.Net.Http/4.700.22.47602
            User-Agent: .NET_Core/3.1.30
            Request-Id: |47ae3bf2-4d058de53621017d.
          }
      +-> Response:
          StatusCode: 404, ReasonPhrase: 'Not Found', Version: 1.1, Content: System.Net.Http.HttpConnectionResponseContent, Headers:
          {
            X-Content-Type-Options: nosniff
            Date: Thu, 03 Nov 2022 19:22:48 GMT
            Content-Type: text/plain; charset=utf-8
            Content-Length: 19
          }
      +-> Response body [length: 19]:
          404 page not found

```

Also when I go to visit the url directly I can see the ssl cert in the browser but also receiving the 404 error.

Is this a config issue with the elastic server?

As far as I can tell my port is open on 8220. I'm not sure what a good next step to take would be. I see nothing on the receiving Elastic Agent / Fleet side so the payload is just not making it over. Meaning no errors to show on the Elastic Agent / Fleet side.

Another line in the debug output for my console app

```auto
} Failed sending event. Events intake API absolute URL: https://<sb-devora-apm-server--url>:8220/intake/v2/events. APM Server response: status code: NotFound, content: 
404 page not found 

```

---

<div class="post-metadata">

**Author:** ![Wolfgang\_Ziegler](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wolfgang_ziegler/32/109929_2.png) [@Wolfgang\_Ziegler](https://discuss.elastic.co/u/Wolfgang_Ziegler)\
**Post date:** [November 3, 2022, 8:33pm UTC](https://discuss.elastic.co/t/troubleshooting-apm-agent-to-fleet-server/318136/2 "2022-11-03T20:33:24Z")

</div>

Hi Daniel,

did you use [NuGet](https://www.nuget.org/packages?q=Elastic.apm) to deploy the APM .NET Agent to your application?  
Can you also check your configuration for me, please?  
At the very minimum you need to configure `ServerUrl` and `SecretToken` (see [documentation](https://www.elastic.co/guide/en/apm/agent/dotnet/current/configuration-on-asp-net-core.html)).

-Wolfgang

---

<div class="post-metadata">

**Author:** ![sb-devora](https://avatars.discourse-cdn.com/v4/letter/s/7c8e57/32.png) [@sb-devora](https://discuss.elastic.co/u/sb-devora)\
**Post date:** [November 3, 2022, 8:48pm UTC](https://discuss.elastic.co/t/troubleshooting-apm-agent-to-fleet-server/318136/3 "2022-11-03T20:48:20Z")

</div>

Thank you for the fast response.

I did not use NuGet for the sample application I'm testing. But I did use it for my main app.

Sample app

```auto
{
  "Logging": {
    "LogLevel": {
      "Default": "Debug",
      "System": "Information",
      "Microsoft": "Information"
    }
  },

  "ElasticApm": {
    "ServerUrls": "https://<sb-devora-fleet--url>:8220",
    "SecretToken": "elk==",
    "VerifyServerCert": false
  },

  "Console": {
    "IncludeScopes": true // Required to use Scopes.
  }
}

```

My app

```auto
	<add key="ElasticApm:ServerUrl" value="https://<sb-devora-fleet--url>:8220" />
	<add key="ElasticApm:SecretToken" value="elk==" />

```

---

<div class="post-metadata">

**Author:** ![Wolfgang\_Ziegler](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wolfgang_ziegler/32/109929_2.png) [@Wolfgang\_Ziegler](https://discuss.elastic.co/u/Wolfgang_Ziegler)\
**Post date:** [November 4, 2022, 7:21am UTC](https://discuss.elastic.co/t/troubleshooting-apm-agent-to-fleet-server/318136/4 "2022-11-04T07:21:47Z")

</div>

Hi Daniel,  
the `404` error suggests a problem with your setup.

If you navigate to _Integrations| APM_

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/3/3/3398e4ec51a4533788fd182b85136014f49e3501.png)

Are you using the correct settings from there?

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/3/a/3a8ced2b18eb63158ab88c591db2abe9973dbbe5.jpeg)

---

<div class="post-metadata">

**Author:** ![sb-devora](https://avatars.discourse-cdn.com/v4/letter/s/7c8e57/32.png) [@sb-devora](https://discuss.elastic.co/u/sb-devora)\
**Post date:** [November 4, 2022, 4:10pm UTC](https://discuss.elastic.co/t/troubleshooting-apm-agent-to-fleet-server/318136/5 "2022-11-04T16:10:11Z")

</div>

Perhaps getting closer?

- Config without VerifyServerCert

```auto
  "ElasticApm": {
    "ServerUrls": "https://xx.xx.xx.xx:8220",
    "SecretToken": "elkxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx==",
    "ServiceName": "SAMPLECONSOLENETCORE",
    "Environment": "production"
  },

```

Provides exception

```auto
System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure.

```

- Config with VerifyServerCert

```auto
  "ElasticApm": {
    "ServerUrls": "https://xx.xx.xx.xx:8220",
    "SecretToken": "elkxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx==",
    "ServiceName": "SAMPLECONSOLENETCORE",
    "Environment": "production",
    "VerifyServerCert": false
  },

```

Provides exception

```auto
      {CentralConfigurationFetcher} Exception was thrown while fetching configuration from APM Server and parsing it. ETag: `<null>'. URL: `https://xx.xx.xx.xx:8220/config/v1/agents?service.name=SAMPLECONSOLENETCORE&service.environment=production'. Apm Server base URL: `https://10.0.2.23:8220/'. WaitInterval: 5m. dbgIterationsCount: 1.
      +-> Request:
          Method: GET, RequestUri: 'https://xx.xx.xx.xx:8220/config/v1/agents?service.name=SAMPLECONSOLENETCORE&service.environment=production', Version: 1.1, Content: <null>, Headers:
          {
            User-Agent: apm-agent-dotnet/1.18.0
            User-Agent: (SAMPLECONSOLENETCORE 1.0.0)
            User-Agent: System.Net.Http/4.700.22.47602
            User-Agent: .NET_Core/3.1.30
            Authorization: [REDACTED]
          }
      +-> Response:
          StatusCode: 404, ReasonPhrase: 'Not Found', Version: 1.1, Content: System.Net.Http.HttpConnectionResponseContent, Headers:
          {
            X-Content-Type-Options: nosniff
            Date: Fri, 04 Nov 2022 15:07:38 GMT
            Content-Type: text/plain; charset=utf-8
            Content-Length: 19
          }
      +-> Response body [length: 19]:
          404 page not found

Elastic.Apm.BackendComm.CentralConfig.CentralConfigurationFetcher+FailedToFetchConfigException: HTTP status code is Not Found (404) which most likely means that APM Server is an old (pre 7.3) version which doesn't support the central configuration endpoint
   at Elastic.Apm.BackendComm.CentralConfig.CentralConfigurationResponseParser.InterpretResponseStatusCode(HttpResponseMessage httpResponse, WaitInfoS waitInfo) in C:\Users\devora\source\pocs\apm-agent-dotnet-main\apm-agent-dotnet-main\src\Elastic.Apm\BackendComm\CentralConfig\CentralConfigurationResponseParser.cs:line 135
   at Elastic.Apm.BackendComm.CentralConfig.CentralConfigurationResponseParser.ParseHttpResponse(HttpResponseMessage httpResponse, String httpResponseBody) in C:\Users\devora\source\pocs\apm-agent-dotnet-main\apm-agent-dotnet-main\src\Elastic.Apm\BackendComm\CentralConfig\CentralConfigurationResponseParser.cs:line 39

```

Should I be able to hit the request urls in a browser as a ssl verification check?

I'm using the self-signed certificate option. When I do visit within the browser I do receive the same 404 type error

---

<div class="post-metadata">

**Author:** ![Wolfgang\_Ziegler](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wolfgang_ziegler/32/109929_2.png) [@Wolfgang\_Ziegler](https://discuss.elastic.co/u/Wolfgang_Ziegler)\
**Post date:** [November 4, 2022, 5:16pm UTC](https://discuss.elastic.co/t/troubleshooting-apm-agent-to-fleet-server/318136/6 "2022-11-04T17:16:57Z")

</div>

Hi Daniel,

can you try a tool like `curl` or `wget` to hit the URL configured at `ElasticApm.ServerUrls`?  
What's the response you get there?

If you have problems with your self-signed certificate, these tools have options like (`--insecure`, or `--no-check-certificate`) that you can try.

If this does not work, there is no change for the .NET APM Agent either to report data to this endpoint.

---

<div class="post-metadata">

**Author:** ![sb-devora](https://avatars.discourse-cdn.com/v4/letter/s/7c8e57/32.png) [@sb-devora](https://discuss.elastic.co/u/sb-devora)\
**Post date:** [November 4, 2022, 8:45pm UTC](https://discuss.elastic.co/t/troubleshooting-apm-agent-to-fleet-server/318136/7 "2022-11-04T20:45:56Z")

</div>

```auto
$ curl https://xx.xx.xx.xx:8220 --insecure
  % Total % Received % Xferd Average Speed Time Time Time Current
                                 Dload Upload Total Spent Left Speed
100 19 100 19 0 0 236 0 --:--:-- --:--:-- --:--:-- 292404 page not found

```

Same 404 issue with curl.

Did I set up my Fleet Server + Elastic Agent incorrectly?

Fleet screen in Kibana shows healthy.  
Elastic-Agent status shows healthy.

Elastic + Kibana + Elastic Agent all v. 8.3.2

Fleet v 1.2.0

---

<div class="post-metadata">

**Author:** ![sb-devora](https://avatars.discourse-cdn.com/v4/letter/s/7c8e57/32.png) [@sb-devora](https://discuss.elastic.co/u/sb-devora)\
**Post date:** [November 4, 2022, 11:07pm UTC](https://discuss.elastic.co/t/troubleshooting-apm-agent-to-fleet-server/318136/8 "2022-11-04T23:07:11Z")

</div>

An update. I switched over to the setup for the legacy APM server and was successful on that approach. I can go with this for now but eventually it seems we will have to migrate over to the Fleet + Agent approach.

---

<div class="post-metadata">

**Author:** ![Wolfgang\_Ziegler](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wolfgang_ziegler/32/109929_2.png) [@Wolfgang\_Ziegler](https://discuss.elastic.co/u/Wolfgang_Ziegler)\
**Post date:** [November 8, 2022, 4:54pm UTC](https://discuss.elastic.co/t/troubleshooting-apm-agent-to-fleet-server/318136/9 "2022-11-08T16:54:21Z")

</div>

It looks like you are using an incorrect URL (at least an incorrect port number) for your APM server URL configuration.  
APM server typically uses port 8200 (while here 8220 is used).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 29, 2022, 12:54pm UTC](https://discuss.elastic.co/t/troubleshooting-apm-agent-to-fleet-server/318136/10 "2022-11-29T12:54:59Z")

</div>

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.
