# Monitoring API value is updated only after logstash restart

**URL:** <https://discuss.elastic.co/t/monitoring-api-value-is-updated-only-after-logstash-restart/100971>\
**Category:** Logstash\
**Created:** [September 19, 2017, 7:08am UTC](https://discuss.elastic.co/t/monitoring-api-value-is-updated-only-after-logstash-restart/100971 "2017-09-19T07:08:07Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![dandrestor](https://avatars.discourse-cdn.com/v4/letter/d/7cd45c/32.png) [@dandrestor](https://discuss.elastic.co/u/dandrestor)\
**Post date:** [September 19, 2017, 7:08am UTC](https://discuss.elastic.co/t/monitoring-api-value-is-updated-only-after-logstash-restart/100971/1 "2017-09-19T07:08:07Z")

</div>

Hi there

I am trying to monitor the number of events that are currently pending in the persisted queue of my Logstash instance. This number seems to update only when I restart Logstash. I looked at the metric in three different places, they all agree:

- The monitoring UI in Kibana (see screenshot below),
- The contents of the `.monitoring` indexes, specifically the `queue.events_count` field, and
- The monitoring API endpoint via curl, specifically the value `pipeline.queue.events`.

The value only changes immediately following a restart of the logstash daemon, and then stays constant:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/b/7/b7382c913c8380b95e5c925c7ad5c5f7dd53f33a.png)

I confirmed the value remained constant even as events from the queue were replayed through to the back-end ES cluster following a network interruption. Am I doing anything wrong, or do I have the wrong expectation?

I am running Logstash 5.5.2 on Linux. The contents of my logstash.yml file:

```
path.data: /var/lib/logstash
path.config: /etc/logstash/conf.d
path.logs: /var/log/logstash
xpack.monitoring.elasticsearch.url: [
  "http://host1:9200",
  "http://host2:9200",
  "http://etc:9200"
]
queue.type: persisted
path.queue: "/path/to/queue"
queue.max_bytes: "200gb"
http.host: "x.x.x.x"
http.port: 9600

```

Thanks,  
Dan

---

<div class="post-metadata">

**Author:** ![jordansissel](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jordansissel/32/44957_2.png) [@jordansissel](https://discuss.elastic.co/u/jordansissel)\
**Post date:** [October 6, 2017, 8:53pm UTC](https://discuss.elastic.co/t/monitoring-api-value-is-updated-only-after-logstash-restart/100971/2 "2017-10-06T20:53:59Z")

</div>

I tested this on Logstash 5.6.1:

```auto
% grep -v '^#' config/logstash.yml
queue.type: persisted

```

```auto
% bin/logstash -e 'input { generator { } } filter { sleep { time => 0.010 } }'

```

```auto
% while true; do ; date | tr '\n' ' ' ; curl -s localhost:9600/_node/stats/pipeline?pretty | jq .pipeline.queue.events; sleep 1; done
Fri Oct 6 13:51:38 PDT 2017 2302534
Fri Oct 6 13:51:39 PDT 2017 2539903
Fri Oct 6 13:51:40 PDT 2017 2539903
Fri Oct 6 13:51:41 PDT 2017 2539903
Fri Oct 6 13:51:42 PDT 2017 2539903
Fri Oct 6 13:51:43 PDT 2017 2539903
Fri Oct 6 13:51:44 PDT 2017 2776786
Fri Oct 6 13:51:45 PDT 2017 2776786
Fri Oct 6 13:51:46 PDT 2017 2776786
Fri Oct 6 13:51:47 PDT 2017 2776786
Fri Oct 6 13:51:48 PDT 2017 3017269
Fri Oct 6 13:51:49 PDT 2017 3017269
Fri Oct 6 13:51:50 PDT 2017 3017269
Fri Oct 6 13:51:51 PDT 2017 3017269
Fri Oct 6 13:51:52 PDT 2017 3017269
Fri Oct 6 13:51:53 PDT 2017 3244089

```

Demonstrated above, the counter is incrementing over time.

It may be an oddity that the counter value updates once every 5-ish seconds, but it certainly is updating for me.

---

<div class="post-metadata">

**Author:** ![dandrestor](https://avatars.discourse-cdn.com/v4/letter/d/7cd45c/32.png) [@dandrestor](https://discuss.elastic.co/u/dandrestor)\
**Post date:** [October 9, 2017, 10:22am UTC](https://discuss.elastic.co/t/monitoring-api-value-is-updated-only-after-logstash-restart/100971/3 "2017-10-09T10:22:08Z")

</div>

Perhaps it's been fixed in the meantime? I will try to upgrade to and test again.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 6, 2017, 10:22am UTC](https://discuss.elastic.co/t/monitoring-api-value-is-updated-only-after-logstash-restart/100971/4 "2017-11-06T10:22:31Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.