Kibana version:
Latest
Elasticsearch version:
Latest
APM Server version:
Latest
APM Agent language and version:
English
Browser version:
Latest
Original install method (e.g. download page, yum, deb, from source, etc.) and version:
Fresh install or upgraded from other version?
Description of the problem including expected versus actual behavior. Please include screenshots (if relevant):
Steps to reproduce:
1.
2.
3.
Errors in browser console (if relevant):
Provide logs and/or server output (if relevant):
[2022-03-25 11:02:50.630 +00:00][Debug] - {AbstractConfigurationReader (FullFrameworkConfigReader)} Service name provided in configuration is WebService_RealTimeHub_ServicesAccessV2
[2022-03-25 11:02:50.658 +00:00][Info] - {AbstractConfigurationReader (FullFrameworkConfigReader)} The agent was started without a service version. The service version will be automatically discovered.
[2022-03-25 11:02:50.659 +00:00][Warning] - {AbstractConfigurationReader (FullFrameworkConfigReader)} Failed to discover service version, the service version will be omitted.
[2022-03-25 11:02:50.686 +00:00][Debug] - {SystemInfoHelper} No /proc/self/cgroup found - the agent will not report container id
[2022-03-25 11:02:50.839 +00:00][Debug] - {BackendCommUtils} Building HTTP client with BaseAddress: apmserver.com:8200/ for BackendCommComponentBase (PayloadSenderV2)...
[2022-03-25 11:02:50.928 +00:00][Debug] - {ExceptionUtils.DoSwallowingExceptions} ElasticApmPayloadSenderV2' (managed ID: 18) thread entry method entered [2022-03-25 11:02:50.956 +00:00][Debug] - {AbstractConfigurationReader (FullFrameworkConfigReader)} GlobalLabels configuration option doesn't have a valid value - using default (empty map) [2022-03-25 11:02:50.960 +00:00][Debug] - {AbstractConfigurationReader (FullFrameworkConfigReader)} MaxQueueEventCount configuration option doesn't have a valid value - using default: 1000 [2022-03-25 11:02:50.962 +00:00][Debug] - {AbstractConfigurationReader (FullFrameworkConfigReader)} MaxBatchEventCount configuration option doesn't have a valid value - using default: 10 [2022-03-25 11:02:50.970 +00:00][Debug] - {AbstractConfigurationReader (FullFrameworkConfigReader)} FlushInterval configuration option doesn't have a valid value - using default: 00:00:10 [2022-03-25 11:02:50.974 +00:00][Debug] - {PayloadSenderV2} Using the following configuration options: Events intake API absolute URL: apmserver.com:8200/intake/v2/events, FlushInterval: 10s, MaxBatchEventCount: 10, MaxQueueEventCount: 1000 [2022-03-25 11:02:51.005 +00:00][Debug] - {BackendCommComponentBase (PayloadSenderV2)} Posted Work loop to internal task scheduler [2022-03-25 11:02:51.023 +00:00][Debug] - {BackendCommComponentBase (PayloadSenderV2)} Waiting for work loop started event... [2022-03-25 11:02:51.027 +00:00][Debug] - {BackendCommComponentBase (PayloadSenderV2)} Signaling work loop started event... [2022-03-25 11:02:51.028 +00:00][Debug] - {BackendCommComponentBase (PayloadSenderV2)} Work loop started signaled [2022-03-25 11:02:51.032 +00:00][Debug] - {ExceptionUtils.DoSwallowingExceptions} BackendCommComponentBase.WorkLoop entered. Current thread: ElasticApmPayloadSenderV2' (managed ID: 18)
[2022-03-25 11:02:51.037 +00:00][Debug] - {BackendCommComponentBase (CentralConfigFetcher)} Disabled - exiting without initializing any members used by work loop
[2022-03-25 11:02:51.039 +00:00][Info] - {CentralConfigFetcher} Central configuration feature is DISABLED because CentralConfig option's value is False (default value is True)
[2022-03-25 11:02:53.288 +00:00][Warning] - {GcMetricsProvider} TraceEventSession initialization failed - GC metrics won't be collected+-> Exception: System.Runtime.InteropServices.COMException: Insufficient system resources exist to complete the requested service. (Exception from HRESULT: 0x800705AA)
at System.Runtime.InteropServices.Marshal.ThrowExceptionForHRInternal(Int32 errorCode, IntPtr errorInfo)
at Microsoft.Diagnostics.Tracing.Session.TraceEventSession.InsureStarted(EVENT_TRACE_PROPERTIES* properties)
at Microsoft.Diagnostics.Tracing.Session.TraceEventSession.get_Source()
at Elastic.Apm.Metrics.MetricsProvider.GcMetricsProvider..ctor(IApmLogger logger, IReadOnlyList`1 disabledMetrics)
[2022-03-25 11:02:53.293 +00:00][Debug] - {CgroupMetricsProvider} /proc/self/cgroup does not exist. Cgroup metrics will not be reported
[2022-03-25 11:02:53.295 +00:00][Info] - {MetricsCollector} Collecting metrics in 30000 milliseconds interval
[2022-03-25 11:02:53.311 +00:00][Debug] - {ElasticApmModule.#1} Found ASP.NET version: 4.8.4465.0
[2022-03-25 11:02:53.319 +00:00][Debug] - {AbstractConfigurationReader (FullFrameworkConfigReader)} CaptureHeaders configuration option doesn't have a valid value - using default: True
[2022-03-25 11:02:53.324 +00:00][Debug] - {HttpDiagnosticListener} Current runtime is detected as Full Framework. RuntimeInformation.FrameworkDescription: .NET Framework 4.8.4470.0
[2022-03-25 11:02:53.354 +00:00][Debug] - {DiagnosticInitializer} Subscribed Elastic.Apm.DiagnosticListeners.HttpDiagnosticListenerFullFrameworkImpl to `System.Net.Http.Desktop' events source
[2022-03-25 11:02:53.355 +00:00][Debug] - {ElasticApmModule.#1} Initialized Agent singleton. .NET runtime: .NET Framework 4.8.4470.0; IIS: 10.0
[2022-03-25 11:02:53.460 +00:00][Debug] - {HttpDiagnosticListenerFullFrameworkImpl} No current transaction, skip creating span for outgoing HTTP request
[2022-03-25 11:02:53.517 +00:00][Debug] - {ElasticApmModule.#1} Incoming request processing started - starting trace...
[2022-03-25 11:02:53.518 +00:00][Debug] - {ElasticApmModule.#1} Incoming request processing started - starting trace...
[2022-03-25 11:02:53.563 +00:00][Debug] - {ElasticApmModule.#1} Incoming request doesn't have elastic-apm-traceparent header - it means request doesn't have incoming distributed tracing data
[2022-03-25 11:02:53.563 +00:00][Debug] - {ElasticApmModule.#1} Incoming request doesn't have elastic-apm-traceparent header - it means request doesn't have incoming distributed tracing data
[2022-03-25 11:02:53.591 +00:00][Debug] - {ElasticApmModule.#1} Incoming request doesn't have valid incoming distributed tracing data - starting trace with new trace ID
[2022-03-25 11:02:53.621 +00:00][Debug] - {ElasticApmModule.#1} Incoming request doesn't have valid incoming distributed tracing data - starting trace with new trace ID
[2022-03-25 11:02:53.655 +00:00][Debug] - {AbstractConfigurationReader (FullFrameworkConfigReader)} No transaction sample rate provided. Defaulting to '1'
[2022-03-25 11:02:53.657 +00:00][Debug] - {AbstractConfigurationReader (FullFrameworkConfigReader)} No transaction sample rate provided. Defaulting to '1'
[2022-03-25 11:02:53.672 +00:00][Debug] - {Tracer} Starting Transaction{Id: 30ed08bacc031592, TraceId: 406fec4d8d9690b22440e61d01c9c8a6, ParentId: null, Name: GET /ServicesAccess.asmx/CheckService, Type: request, Outcome: Unknown, IsSampled: True}
[2022-03-25 11:02:53.673 +00:00][Debug] - {Tracer} Starting Transaction{Id: 05ccb8d135e98c41, TraceId: 1b6c1be78252b2483d52edb8a0c67b87, ParentId: null, Name: GET /, Type: request, Outcome: Unknown, IsSampled: True}
[2022-03-25 11:02:53.802 +00:00][Warning] - {PayloadSenderV2} Failed reading APM server info+-> Exception: System.Net.Http.HttpRequestException: An error occurred while sending the request.
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Net.Http.HttpClient.d__58.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Elastic.Apm.ServerInfo.ApmServerInfoProvider.d__0.MoveNext()
[2022-03-25 11:02:53.919 +00:00][Debug] - {ElasticApmModule.#1} Incoming request processing started - starting trace...
[2022-03-25 11:02:53.921 +00:00][Debug] - {ElasticApmModule.#1} Incoming request doesn't have elastic-apm-traceparent header - it means request doesn't have incoming distributed tracing data
[2022-03-25 11:02:53.922 +00:00][Debug] - {ElasticApmModule.#1} Incoming request doesn't have valid incoming distributed tracing data - starting trace with new trace ID
[2022-03-25 11:02:53.922 +00:00][Debug] - {AbstractConfigurationReader (FullFrameworkConfigReader)} No transaction sample rate provided. Defaulting to '1'
[2022-03-25 11:02:53.923 +00:00][Debug] - {Tracer} Starting Transaction{Id: db151a6c030d8d5d, TraceId: f2e4a6506e27181dc7c30e98214951e8, ParentId: null, Name: GET /, Type: request, Outcome: Unknown, IsSampled: True}
[2022-03-25 11:02:53.927 +00:00][Debug] - {ElasticApmModule.#1} Incoming request processing finished - ending trace...
[2022-03-25 11:02:53.953 +00:00][Debug] - {PayloadSenderV2} Enqueued Transaction. newEventQueueCount: 1. MaxQueueEventCount: 1000. Transaction: Transaction{Id: db151a6c030d8d5d, TraceId: f2e4a6506e27181dc7c30e98214951e8, ParentId: null, Name: GET /, Type: request, Outcome: Success, IsSampled: True}.
[2022-03-25 11:02:53.976 +00:00][Debug] - {ElasticApmModule.#1} Incoming request processing finished - ending trace...
[2022-03-25 11:02:53.978 +00:00][Debug] - {PayloadSenderV2} Enqueued Transaction. newEventQueueCount: 2. MaxQueueEventCount: 1000. Transaction: Transaction{Id: 05ccb8d135e98c41, TraceId: 1b6c1be78252b2483d52edb8a0c67b87, ParentId: null, Name: GET /, Type: request, Outcome: Success, IsSampled: True}.
[2022-03-25 11:02:54.472 +00:00][Debug] - {ElasticApmModule.#1} Incoming request processing finished - ending trace...
[2022-03-25 11:02:54.473 +00:00][Debug] - {PayloadSenderV2} Enqueued Transaction. newEventQueueCount: 3. MaxQueueEventCount: 1000. Transaction: Transaction{Id: 30ed08bacc031592, TraceId: 406fec4d8d9690b22440e61d01c9c8a6, ParentId: null, Name: GET /ServicesAccess.asmx/CheckService, Type: request, Outcome: Success, IsSampled: True}.
[2022-03-25 11:03:04.075 +00:00][Warning] - {PayloadSenderV2} Failed sending events. Following events were not transferred successfully to the server (apmserver.com:8200/):
Transaction{Id: db151a6c030d8d5d, TraceId: f2e4a6506e27181dc7c30e98214951e8, ParentId: null, Name: GET /, Type: request, Outcome: Success, IsSampled: True},
Transaction{Id: 05ccb8d135e98c41, TraceId: 1b6c1be78252b2483d52edb8a0c67b87, ParentId: null, Name: GET /, Type: request, Outcome: Success, IsSampled: True},
Transaction{Id: 30ed08bacc031592, TraceId: 406fec4d8d9690b22440e61d01c9c8a6, ParentId: null, Name: GET /ServicesAccess.asmx/CheckService, Type: request, Outcome: Success, IsSampled: True}+-> Exception: System.Net.Http.HttpRequestException: An error occurred while sending the request.
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Net.Http.HttpClient.d__58.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Elastic.Apm.Report.PayloadSenderV2.d__32.MoveNext()
[2022-03-25 11:03:08.413 +00:00][Debug] - {ElasticApmModule.#1} Incoming request processing started - starting trace...
[2022-03-25 11:03:08.415 +00:00][Debug] - {ElasticApmModule.#1} Incoming request doesn't have elastic-apm-traceparent header - it means request doesn't have incoming distributed tracing data
[2022-03-25 11:03:08.416 +00:00][Debug] - {ElasticApmModule.#1} Incoming request doesn't have valid incoming distributed tracing data - starting trace with new trace ID
[2022-03-25 11:03:08.418 +00:00][Debug] - {AbstractConfigurationReader (FullFrameworkConfigReader)} No transaction sample rate provided. Defaulting to '1'
[2022-03-25 11:03:08.419 +00:00][Debug] - {Tracer} Starting Transaction{Id: 5fdc39804feaa707, TraceId: 9161b74e08059b2b492e0dd5a2fb6b24, ParentId: null, Name: GET /ServicesAccess.asmx/CheckService, Type: request, Outcome: Unknown, IsSampled: True}
[2022-03-25 11:03:08.421 +00:00][Debug] - {ElasticApmModule.#1} Incoming request processing finished - ending trace...
[2022-03-25 11:03:08.422 +00:00][Debug] - {PayloadSenderV2} Enqueued Transaction. newEventQueueCount: 1. MaxQueueEventCount: 1000. Transaction: Transaction{Id: 5fdc39804feaa707, TraceId: 9161b74e08059b2b492e0dd5a2fb6b24, ParentId: null, Name: GET /ServicesAccess.asmx/CheckService, Type: request, Outcome: Success, IsSampled: True}.
[2022-03-25 11:03:08.641 +00:00][Debug] - {ElasticApmModule.#1} Incoming request processing started - starting trace...
[2022-03-25 11:03:08.644 +00:00][Debug] - {ElasticApmModule.#1} Incoming request doesn't have elastic-apm-traceparent header - it means request doesn't have incoming distributed tracing data
[2022-03-25 11:03:08.646 +00:00][Debug] - {ElasticApmModule.#1} Incoming request doesn't have valid incoming distributed tracing data - starting trace with new trace ID
[2022-03-25 11:03:08.648 +00:00][Debug] - {AbstractConfigurationReader (FullFrameworkConfigReader)} No transaction sample rate provided. Defaulting to '1'
[2022-03-25 11:03:08.649 +00:00][Debug] - {Tracer} Starting Transaction{Id: f95023d98a119c18, TraceId: 6a9dcd252850d0a0fe677e2e7cc506b8, ParentId: null, Name: GET /ServicesAccess.asmx/CheckService, Type: request, Outcome: Unknown, IsSampled: True}
[2022-03-25 11:03:08.649 +00:00][Debug] - {ElasticApmModule.#1} Incoming request processing finished - ending trace...
[2022-03-25 11:03:08.650 +00:00][Debug] - {PayloadSenderV2} Enqueued Transaction. newEventQueueCount: 2. MaxQueueEventCount: 1000. Transaction: Transaction{Id: f95023d98a119c18, TraceId: 6a9dcd252850d0a0fe677e2e7cc506b8, ParentId: null, Name: GET /ServicesAccess.asmx/CheckService, Type: request, Outcome: Success, IsSampled: True}.
[2022-03-25 11:03:14.148 +00:00][Warning] - {PayloadSenderV2} Failed sending events. Following events were not transferred successfully to the server (apmserver.com:8200/):
Transaction{Id: 5fdc39804feaa707, TraceId: 9161b74e08059b2b492e0dd5a2fb6b24, ParentId: null, Name: GET /ServicesAccess.asmx/CheckService, Type: request, Outcome: Success, IsSampled: True},
Transaction{Id: f95023d98a119c18, TraceId: 6a9dcd252850d0a0fe677e2e7cc506b8, ParentId: null, Name: GET /ServicesAccess.asmx/CheckService, Type: request, Outcome: Success, IsSampled: True}+-> Exception: System.Net.Http.HttpRequestException: An error occurred while sending the request.
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Net.Http.HttpClient.d__58.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Elastic.Apm.Report.PayloadSenderV2.d__32.MoveNext()
