Remaining Transactions!

[Tubby_Stubby]

Kibana version: 8.14.1

Elasticsearch version: 8.14.1

APM Server version: 8.14.1

APM Agent language and version: elastic-apm-node@4.7.0

Original install method (e.g. download page, yum, deb, from source, etc.) and version: apt/deb

Fresh install or upgraded from other version?: Fresh

Is there anything special in your setup?
Updated the ILM policy by creating traces-apm@custom component template

Description of the problem including expected versus actual behavior. Please include screenshots (if relevant):
We have 200+ APIs but not all are shown in the UI instead an entry comes as Remaining Transactions.

The docs said to reduce the cardinality of transaction names, but we don't add transaction names manually instead they are being captured by the default express instrumentation. Also, I am not able to find what these transactions are as I can't see the full url either.

How should I proceed from here?

[trentm (Trent Mick)]

Hi @Tubby_Stubby,

That "Transactions" view is just for a single service, right?
For example, here I have a "example-trace-hapi" service with one transaction:

When you say "We have 200+ APIs" ... do you mean you have 200+ separate endpoints on a single service? I.e., you have an Express app that has more than 200 separate routes?

---

This is a guess: are you using the usePathAsTransactionName configuration option with the Node.js APM agent? That can lead to transaction.name cardinality issues.

---

You should be able to search in "Discover" in Kibana for all transaction documents for your service. Select the "APM" index alias, search for service.name: "your-service-name" and processor.event: "transaction", and add the transaction.name column. E.g. here is me doing it for my small "example-trace-hapi" service:

Does that help get you further?
(If you have a support contract and are using Elastic Cloud, you could contact support, which would allow us -- with permission -- to look at your APM data to see if some issue sticks out.)

[Tubby_Stubby]

Thanks for detailed response.

@trentm

When you say "We have 200+ APIs" ... do you mean you have 200+ separate endpoints on a single service? I.e., you have an Express app that has more than 200 separate routes?

Yes. An Express app with 200+ endpoints.

@trentm

This is a guess: are you using the usePathAsTransactionName configuration option with the Node.js APM agent? That can lead to transaction.name cardinality issues.

No.

I checked discover for transaction names and I can see all of them, confirmed the total count of unique transactions in lens apart from some Unknow which I think I can debug from discover. Earlier I was not filtering for event type processor.event: "transaction", so was seeing entries without any names which had me little confused.

Is there any way to show to all transactions in Transaction view without the 200 limit?

Currently we are just trying out elastic with manual deployment. Will definitely checkout Elastic Cloud.

[trentm (Trent Mick)]

Sorry for the delay in responding.

@Tubby_Stubby

Is there any way to show to all transactions in Transaction view without the 200 limit?

IIUC the limit should actually be 1000.
It is a hardcoded number here in Kibana code, FWIW: kibana/x-pack/plugins/observability_solution/apm/server/routes/services/get_service_transaction_groups.ts at 9557e064a1b0c9768bb3252db8395bc943136054 · elastic/kibana · GitHub

@Tubby_Stubby

I checked discover for transaction names and I can see all of them

Did you find more than 1000?

[Tubby_Stubby]

@trentm

IIUC the limit should actually be 1000.

Got it, seems pretty reasonable.

@trentm

Did you find more than 1000?

Nope the total is in between 250-300.