MIssing transactions in APM UI

Elastic Observability APM
1

I'm using the APM Python agent with a Django application. After migrating from Opbeat to Elastic APM I've been missing transactions from many Django views in the views table (in the APM UI in Kibana). They are, however, shown when explicitly filtered for and they are always represented in the graph showing requests per minute (unless filtered out). It did seem that all affected views were Django REST Framework views but that might not be true. Not all but the vast majority of the missing transactions have a HTTP 4xx result. The views with missing transactions should have very high impact since there are thousands of requests per minute for those ones, in contrast to many other views.

To me, this appears to be a bug in the APM UI as all transactions still are indexed and available when filtered or searched for.

Oh, I'm running Elasticsearch, Kibana and APM versions 6.4.2. The Python agent is 2.1.1 but I guess that shouldn't be an issue.

2

Hi David,

The APM UI will group transactions by name, and display the top 100 (ordered by desc avg. response time). If you have more than 100 unique transaction names (which in your case translates to django views), you will not see all of them.

This could be the problem you are seeing. To determine how many unique transaction names you have, please run the following query in Dev Tools:

GET apm*/_search
{
    "size" : 0,
    "aggs" : {
        "distinct_names" : {
            "cardinality" : {
              "field" : "transaction.name.keyword"
            }
        }
    }
}

Let me know what count it returns, and we can take it from there.

3

The query returns a count of 302 unique transaction names.

The queries that I'm missing have a really low average response time in comparison to those that are visible in the table. The thing is however, in Opbeat the missing queries were at the top of the table where I expected them to be. When migrating from Opbeat I expected the same result from Elastic APM.

4

Okay, so it sounds like the problem is with the number of unique transaction names.
There is unfortunately no way to increase the number of transaction groups but I have created an issue here to make it configurable.

I'm curious why you have such a high number of unique transaction names. Could there be an issue with the grouping logic?

As an example, the urls "/users/42" and "/users/1337" should have the transaction name: "/users/:userId".

5

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.