Elastic Ruby on Rails APM: Queue is full issue

Elastic Observability APM
1

Hi Team

I'm using APM version: 7.8.0 for ruby on rails app. I integrated this with prod and dev.

However, for some reason I'm not able to see recent requests for prod site.

Here are the logs in my app. I see queue is full

Screen Shot 2020-08-28 at 11.09.48 AM
Screen Shot 2020-08-28 at 11.09.48 AM880×268 118 KB

Any suggestions on Why this is happening ? bcos my APM server is working fine and I'm able to get data from Dev.

Here are the logs of APM server:

|2020-08-28T16:13:07.117Z|INFO|[request]|middleware/log_middleware.go:97|request ok|{"request_id": "9557d79b-09c4-409a-bbab-0f288f193049", "method": "GET", "URL": "/config/v1/agents?service.name=Ilearn", "content_length": 0, "remote_address": "10.85.77.124", "user-agent": "elastic-apm-ruby/3.10.1 http.rb/4.4.1 ruby/2.4.1", "response_code": 200}|
|---|---|---|---|---|---|
|2020-08-28T16:13:07.847Z|INFO|[request]|middleware/log_middleware.go:97|request ok|{"request_id": "d77adb92-c9b2-4cd2-ae57-dea86d206a5a", "method": "GET", "URL": "/config/v1/agents?service.name=IlearnDev", "content_length": 0, "remote_address": "10.85.77.118", "user-agent": "elastic-apm-ruby/3.10.1 http.rb/4.4.1 ruby/2.4.1", "response_code": 200}|
|2020-08-28T16:13:13.253Z|INFO|[request]|middleware/log_middleware.go:97|request accepted|{"request_id": "2b7a1c36-430e-44fc-98cf-76df7304a0e7", "method": "POST", "URL": "/intake/v2/events", "content_length": 558, "remote_address": "10.213.180.192", "user-agent": "elastic-apm-ruby/3.10.1 http.rb/4.4.1 ruby/2.4.1", "response_code": 202}|
|2020-08-28T16:13:20.814Z|INFO|[request]|middleware/log_middleware.go:97|request ok|{"request_id": "61dddcac-6157-4afb-bc92-6f592f262d2b", "method": "GET", "URL": "/config/v1/agents?service.name=IlearnDev", "content_length": 0, "remote_address": "10.213.180.192", "user-agent": "elastic-apm-ruby/3.10.1 http.rb/4.4.1 ruby/2.4.1", "response_code": 200}|
|2020-08-28T16:13:20.935Z|INFO|[request]|middleware/log_middleware.go:97|request accepted|{"request_id": "4e4e7706-7a07-4980-aab3-fc238ca2d3e6", "method": "POST", "URL": "/intake/v2/events", "content_length": 608, "remote_address": "10.85.77.118", "user-agent": "elastic-apm-ruby/3.10.1 http.rb/4.4.1 ruby/2.4.1", "response_code": 202}|

I'm trying to understand what made the queue full ? bcos queue can send data to apm server and apm server is not busy.

Please suggest

Thanks
Rahul

2

Hi Team

Any suggestions would be helpful.

Thank you

3

Hope this link can can help
APM Server uses an internal queue to buffer incoming events. A larger queue can retain more data if Elasticsearch is unavailable for longer periods, and it alleviates problems that might result from sudden spikes of traffic. You can adjust the queue size by overriding queue.mem.events . Increasing queue.mem.events can significantly affect APM Server memory usage.

Also this link can help

4

Sure @ylasri

I will have a look.

Thank you : )

5

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.