# Logstash persisted queue with xpack monitoring

**URL:** <https://discuss.elastic.co/t/logstash-persisted-queue-with-xpack-monitoring/210898>\
**Category:** Logstash\
**Created:** [December 6, 2019, 1:25pm UTC](https://discuss.elastic.co/t/logstash-persisted-queue-with-xpack-monitoring/210898 "2019-12-06T13:25:45Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Yungyoung\_Ok](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yungyoung_ok/32/43465_2.png) [@Yungyoung\_Ok](https://discuss.elastic.co/u/Yungyoung_Ok)\
**Post date:** [December 6, 2019, 1:25pm UTC](https://discuss.elastic.co/t/logstash-persisted-queue-with-xpack-monitoring/210898/1 "2019-12-06T13:25:45Z")

</div>

I am using x-pack monitoring to monitor the logstash.

logstash.yml  
xpack.monitoring.enabled: true  
xpack.monitoring.elasticsearch.hosts: ["node-1", "node-2", "node-3"]

my logstash command:  
./bin/logstash -f my\_conf.conf --pipeline.id logstash-forwarder --debug

result:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/b/6/b6506beebb0d289999bdd425104e32b568702d3a.png)

Because of the possibility of data loss today, i added a persisted queue and dead\_letter\_queue.  
like this:  
queue.type: persisted  
dead\_letter\_queue.enable: true

**No other settings were modified at all.**  
**And run with the same command.**

result:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/b/6/b6315080cf67c61ce05dd2811f5ac1b9bef7a597.png)

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/7/a/7a4ff548d0124186c01b74948562310866b89b22.png)

**The added setting is well applied, but the monitoring is not applied.**  
**Do i have any settings to add in order to use this function?**

---

<div class="post-metadata">

**Author:** ![chrisronline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrisronline/32/28230_2.png) [@chrisronline](https://discuss.elastic.co/u/chrisronline)\
**Post date:** [December 9, 2019, 7:57pm UTC](https://discuss.elastic.co/t/logstash-persisted-queue-with-xpack-monitoring/210898/2 "2019-12-09T19:57:58Z")

</div>

Hi @Yungyoung_Ok,

That is very strange. Just to double check, can you paste the full `my_conf.conf` file?

---

<div class="post-metadata">

**Author:** ![Yungyoung\_Ok](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yungyoung_ok/32/43465_2.png) [@Yungyoung\_Ok](https://discuss.elastic.co/u/Yungyoung_Ok)\
**Post date:** [December 10, 2019, 1:53am UTC](https://discuss.elastic.co/t/logstash-persisted-queue-with-xpack-monitoring/210898/3 "2019-12-10T01:53:11Z")

</div>

sure

1. logstash.yml  

2. my\_conf.conf  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/9/0/90984363d8073f6d3a27f9403a691ec5d74b9d03.png)

3. command  
./bin/logstash -f my\_conf.conf --pipeline.id logstash-forwarder --debug

that's all.

---

<div class="post-metadata">

**Author:** ![chrisronline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrisronline/32/28230_2.png) [@chrisronline](https://discuss.elastic.co/u/chrisronline)\
**Post date:** [December 10, 2019, 2:23am UTC](https://discuss.elastic.co/t/logstash-persisted-queue-with-xpack-monitoring/210898/4 "2019-12-10T02:23:46Z")

</div>

What version of logstash are you using?

Per [the docs](https://www.elastic.co/guide/en/logstash/current/running-logstash-command-line.html), the `-f` option overrides the logstash config, which means it's attempting to parse the `my_conf.conf` as as `logstash.yml`

Maybe adding `--config.debug` might help clear up which file is being used?

---

<div class="post-metadata">

**Author:** ![Yungyoung\_Ok](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yungyoung_ok/32/43465_2.png) [@Yungyoung\_Ok](https://discuss.elastic.co/u/Yungyoung_Ok)\
**Post date:** [December 11, 2019, 1:41pm UTC](https://discuss.elastic.co/t/logstash-persisted-queue-with-xpack-monitoring/210898/5 "2019-12-11T13:41:39Z")

</div>

logstash version: 7.4.2

command:  
./bin/logstash -f my\_conf.conf --debug --config.debug

result:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/3/5/350b23f8eca185582ffd4fc0c7bb1f8146d3a5e9.png)

> Per [the docs](https://www.elastic.co/guide/en/logstash/current/running-logstash-command-line.html), the `-f` option overrides the logstash config, which means it's attempting to parse the `my_conf.conf` as as `logstash.yml`

what's mean?  
I know, '-f option' specifies the logstash file to execute.  
Is the option related to logstash.yml?  
Isn't the '--path.settings option' related option to logstash.yml?

The debugging output configuration file looks at the config directory in the default path,  
All settings except the monitoring will have the changed values.

What should I do more?

---

<div class="post-metadata">

**Author:** ![chrisronline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrisronline/32/28230_2.png) [@chrisronline](https://discuss.elastic.co/u/chrisronline)\
**Post date:** [December 11, 2019, 2:46pm UTC](https://discuss.elastic.co/t/logstash-persisted-queue-with-xpack-monitoring/210898/6 "2019-12-11T14:46:32Z")

</div>

Ah, yes - your understanding is correct.

If you go and remove those two settings:

```auto
queue.type: persisted
dead_letter_queue.enable: true

```

Does it properly recognize the `xpack.monitoring.enabled` setting?

---

<div class="post-metadata">

**Author:** ![Yungyoung\_Ok](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yungyoung_ok/32/43465_2.png) [@Yungyoung\_Ok](https://discuss.elastic.co/u/Yungyoung_Ok)\
**Post date:** [December 16, 2019, 2:27am UTC](https://discuss.elastic.co/t/logstash-persisted-queue-with-xpack-monitoring/210898/7 "2019-12-16T02:27:33Z")

</div>

of course.

---

<div class="post-metadata">

**Author:** ![chrisronline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrisronline/32/28230_2.png) [@chrisronline](https://discuss.elastic.co/u/chrisronline)\
**Post date:** [December 16, 2019, 5:26pm UTC](https://discuss.elastic.co/t/logstash-persisted-queue-with-xpack-monitoring/210898/8 "2019-12-16T17:26:21Z")

</div>

Hmm. I'm not seeing this behavior locally and not sure about the details of logstash (I'm on the stack monitoring team, which is tagged here).

I don't think this is an issue with how monitoring works, but rather something around logstash configs.. I'm going to remove the `stack-monitoring` tag and a member of the Logstash team will pick this up and help more

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 13, 2020, 5:26pm UTC](https://discuss.elastic.co/t/logstash-persisted-queue-with-xpack-monitoring/210898/9 "2020-01-13T17:26:22Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
