No services found in apm dashboard in kibana

[mvnov (Marv__Ch__)]

Kibana version:7.4.2

Elasticsearch version:74.2

APM Server version:7.4.2

APM Agent language and version: .net , Elastic.Apm.NetCoreAll , 1.5.1

Browser version: chrome, Version 85.0.4183.83 (Official Build) (64-bit)

Original install method (e.g. download page, yum, deb, from source, etc.) and version: docker

Fresh install or upgraded from other version? fresh install

Is there anything special in your setup? For example, are you using the Logstash or Kafka outputs? Are you using a load balancer in front of the APM Servers? Have you changed index pattern, generated custom templates, changed agent configuration etc.
No logstash/kafks, data from elastic.apm agent directly. No custom templates. ServiceUrls and ServiceName configured only under ElasticApm section in appsettings.json, SecretToken is commented out.

Description of the problem including expected versus actual behavior. Please include screenshots (if relevant):
We have elasticsearch 7.4.2 and kibana 7.4.2 installed from docker for system logging and they work fine. I added elastic apm server recently from the docker image and it starts with the service url of the elasticsearch server. From the log, I think the apm server started correctly.

in kibana, I checked apm server status, it was fine

and the agents section, I selected .net tab, and it looked also fine

BTW, when loading kibana objects, there was a warning and I overwrote it

everything looked good and I went to apm doashboard, a few http requests were sent but there was no services found ,not traces

and I could see logs under "apm-*" in discover page

Steps to reproduce:
1.
2.
3.

Errors in browser console (if relevant):

Provide logs and/or server output (if relevant):

[mvnov (Marv__Ch__)]

looking at the logs from apm server

did they mean the apm data was not written to elastic server?

[simitt (Silvia Mitter)]

These are expected log lines on level info and mean that requests from agents have been accepted and are processed.
Are you experiencing any issues with data not showing up in Kibana?

[mvnov (Marv__Ch__)]

yes, no services found, shouldn't there be one service listed? the service with elastic.apm agent installed was actually sending data to apm server. am I missing something during the configuration? anything that I can check/verify?

[simitt (Silvia Mitter)]

Ah sorry I missed that originally. As a next step we should identify if data are not ingested as expected or they cannot be shown for some reason. From the APM Server logs above I'd expect data being ingested. If you can confirm that, please take a look at the APM troubleshooting section in Kibana.

[mvnov (Marv__Ch__)]

Yes, the data is ingested. I also can see apm-* index pattern is created in Management/Index patterns page.

But I noticed there are confllic fields

look into one of them

I just deleted all apm-* .apm-* indices and remove apm-* index pattern from kibana, and remove the apm-server docker container, then relaunched a docker container with the apm-server docker image again (same script), then I see the expected service listed in apm page

it's now working like a charm ..

thanks simitt