Nothing to see in Traces

[FutuPoul (Poul)]

Hello,

I have two services that communicate a->b. The information appears in the "Trace sample" but nothing appears on the "Traces" . Details on the screenshots. I am asking for advice as to what may be incorrect.

Best Regards
Poul

[dgieselaar (Dario Gieselaar)]

Hello @FutuPoul! The list of traces shows root transactions, ie transactions that do not have a parent 1. My only guess right now is that all of the transactions somehow have a parent. To figure that out, can you run the following query to ES (from the Kibana Dev Tools Console or directly to ES) and show me the results (please remove any sensitive information):

GET apm-*/_search
{
  "query": {
    "bool": {
      "filter": [
        {
          "range": {
            "@timestamp": {
              "gte": "now-24h"
            }
          }
        },
        {
          "term": {
            "processor.event": "transaction"
          }
        }
      ],
      "must_not": [
        {
          "exists": {
            "field": "parent.id"
          }
        }
      ]
    }
  }
}

[FutuPoul (Poul)]

 "gte": "now-90h"

result

{
  "took" : 18,
  "timed_out" : false,
  "_shards" : {
    "total" : 16,
    "successful" : 16,
    "skipped" : 7,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 0,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  }
}

[dgieselaar (Dario Gieselaar)]

Thanks @FutuPoul! Looks like there are no root transactions, which is what we need to populate the trace UI. Do you have any idea how that could happen?

[GregKalapos]

Hi @FutuPoul,

we discussed this with @dgieselaar and this seems to be caused by the .NET APM Agent. There is already an issue here with more details - that issue already has 1 theory on how this could happen.

One useful info would be the value of the parent.id in those cases - one easy way to get it would be to just select a transaction, click the "View full trace" if it's active (basically your 2. screenshot), select the 1. transaction (in the screenshot it'd be the 1. green box with HTTP 2xx POST Example/Hello), then click the "Actions" button, then click "View sample document", then you should see the parent.id there. It'd be very useful for me to see the format of it (this could be an issue with Activity which has a different id format - this could be verified by seeing the parent.id value).

Nevertheless I'm already working on this and try to figure out a solution.

[FutuPoul (Poul)]

View full trace is inactive

 "parent": {
      "id": "|988db67b-4e2219252ff5676f."
    },

My environment:
Elastic.Apm.NetCoreAll v1.5.1
.Net Core 3.1

[FutuPoul (Poul)]

do You know when the problem will be solved?

[GregKalapos]

Thanks @FutuPoul,

this is very useful info. Based on the format of that parent.id it looks this comes from System.Diagnostics.Activity - with that, at least I know how and why this happens.

I can't tell you when it will be solved, but this is fairly high priority - best is to follow the GitHub issue.

[FutuPoul (Poul)]

ok
Thank you for your help