Hi All,
Approx from 10-15 days, I am continuously facing graylog web-interface down problem. First it's slow down and then after it finally went down by throwing 503 error.
Here are the logs that I am getting:
2018-06-20 11:51:24,205 - [ERROR] - from org.graylog2.restclient.lib.ApiClient in pool-74-thread-1
API call Interrupted
java.lang.InterruptedException: null
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1039) ~[na:1.8.0_25]
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328) ~[na:1.8.0_25]
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) ~[na:1.8.0_25]
at com.ning.http.client.providers.netty.future.NettyResponseFuture.get(NettyResponseFuture.java:158) ~[com.ning.async-http-client-1.9.31.jar:na]
at org.graylog2.restclient.lib.ApiClientImpl$ApiRequestBuilder.executeOnAll(ApiClientImpl.java:608) ~[org.graylog2.graylog2-rest-client--1.3.0-1.3.0.jar:na]
at controllers.api.MetricsController$PollingJob.run(MetricsController.java:117) [graylog-web-interface.graylog-web-interface-1.3.0.jar:1.3.0]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_25]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_25]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_25]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_25]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_25]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_25]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_25]
2018-06-20 11:51:24,215 - [ERROR] - from org.graylog2.restclient.lib.ApiClient in pool-73-thread-1
API call Interrupted
java.lang.InterruptedException: null
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1039) ~[na:1.8.0_25]
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328) ~[na:1.8.0_25]
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) ~[na:1.8.0_25]
at com.ning.http.client.providers.netty.future.NettyResponseFuture.get(NettyResponseFuture.java:158) ~[com.ning.async-http-client-1.9.31.jar:na]
at org.graylog2.restclient.lib.ApiClientImpl$ApiRequestBuilder.executeOnAll(ApiClientImpl.java:608) ~[org.graylog2.graylog2-rest-client--1.3.0-1.3.0.jar:na]
at controllers.api.MetricsController$PollingJob.run(MetricsController.java:117) [graylog-web-interface.graylog-web-interface-1.3.0.jar:1.3.0]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_25]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_25]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_25]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_25]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_25]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_25]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_25]
Elasticsearch Version that I am using - "1.3.4"
Graylog-web-interface version - "1.3.0"
Thanks
Tafsir Alam