{PayloadSenderV2} Failed sending events. Following events were not transferred successfully to the server

Elastic Observability APM
1

Kibana version :
7.4.0 [Cloud]

Elasticsearch version :
7.4.0 [Cloud]

APM Server version :
7.4.0

APM Agent language and version :
dotnet 1.2

Browser version :
N/A

Original install method (e.g. download page, yum, deb, from source, etc.) and version :
7.0.0 -> 7.2.0 -> 7.4.0 through deb

Fresh install or upgraded from other version?
7.0.0 -> 7.2.0 -> 7.4.0 through deb

Is there anything special in your setup? For example, are you using the Logstash or Kafka outputs? Are you using a load balancer in front of the APM Servers? Have you changed index pattern, generated custom templates, changed agent configuration etc.

NGINX reverse proxy in front of APM Server and between APM Server and Logstash server

Description of the problem including expected versus actual behavior. Please include screenshots (if relevant) :

We have an issue with our APM dotnet agents (1.2) which intermittently shows transfer failures of a specific transaction. Most of them are transferred fine but about 10% is failing. The mentioned TraceIds are not found in the APM indexes.

	{PayloadSenderV2} Failed sending events. Following events were not transferred successfully to the server (https://xxxx/):
    Elastic.Apm.Metrics.MetricSet,
    Transaction{Id: 334a2fe2352bf342, TraceId: 5367449ab20145e9bc8074ef2d784b47, ParentId: null, Name: GET Preference/HealthCheck, Type: request, IsSampled: True},
    Elastic.Apm.Metrics.MetricSet,
    Elastic.Apm.Metrics.MetricSet,
    Transaction{Id: 1b2f3a30bc4fa415, TraceId: f0218af56d117708ffcf30e1dd932a5e, ParentId: null, Name: GET Preference/HealthCheck, Type: request, IsSampled: True},
    Elastic.Apm.Metrics.MetricSet,
    Transaction{Id: 7900165db05c20bb, TraceId: fee7c606f1c28503c32fa991774f7cab, ParentId: null, Name: GET Preference/HealthCheck, Type: request, IsSampled: True},
    Elastic.Apm.Metrics.MetricSet

Is an empty ParentId an issue? Where should investigate to find out under what circumstances the transfer fails?

2

Hi @J_Weeda

Can you please also show us the log lines after the last line from what you pasted Elastic.Apm.Metrics.MetricSet - we print the exception itself also, that would be after the payload (the Transactions, Spans, and Metrics), I'd hope that reveals more info.

To your question:

Is an empty ParentId an issue?

No, that should not be a problem - it just means it's the 1. transaction in a trace, that can be normal.

3

thanks, @GregKalapos, I don't see any exceptions listed below the pasted messages. Having spot below, I will double check the package reference. Any other ideas why there is no exception listed?

4

I will double check the package reference. Any other ideas why there is no exception listed?

Actually double checking the package reference is a good idea. I looked into the changes around here and we had some commits modifying the logging. One thing that comes to my mind is that you maybe have an older agent version. Could you please double check if this is really 1.2 (latest)?

The latest should definitely log an exception.

5

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.