# Logstash monitoring vs management

**URL:** <https://discuss.elastic.co/t/logstash-monitoring-vs-management/209387>\
**Category:** Logstash\
**Tags:** elastic-stack-monitoring, fleet\
**Created:** [November 25, 2019, 10:08pm UTC](https://discuss.elastic.co/t/logstash-monitoring-vs-management/209387 "2019-11-25T22:08:41Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![ibexit](https://avatars.discourse-cdn.com/v4/letter/i/a88e4f/32.png) [@ibexit](https://discuss.elastic.co/u/ibexit)\
**Post date:** [November 25, 2019, 10:08pm UTC](https://discuss.elastic.co/t/logstash-monitoring-vs-management/209387/1 "2019-11-25T22:08:41Z")

</div>

using docker-compose I´m able to start and monitor my logstash instance with this environment settings:

```
- XPACK_SECURITY_ENABLED=true
- XPACK.MONITORING.ENABLED=true
- XPACK.MONITORING.ELASTICSEARCH.HOSTS=["https://192.168.178.100:9200"]
- XPACK.MONITORING.ELASTICSEARCH.USERNAME=logstash_system
- XPACK.MONITORING.ELASTICSEARCH.PASSWORD=***
- XPACK.MONITORING.ELASTICSEARCH.SSL.CERTIFICATE_AUTHORITY=/usr/share/logstash/certs/ca/ca.crt
- XPACK.MONITORING.ELASTICSEARCH.SSL.VERIFICATION_MODE=certificate

```

I can see the the logstash in kibana/monitoring. But enabling the central pipeline management feature breaks the monitoring (the management works). This is the additional environment variables causing the trouble:

```
- XPACK.MANAGEMENT.ENABLED=true
- XPACK.MANAGEMENT.ELASTICSEARCH.HOSTS=["https://192.168.178.100:9200"]
- XPACK.MANAGEMENT.ELASTICSEARCH.USERNAME=logstash_user
- XPACK.MANAGEMENT.ELASTICSEARCH.PASSWORD=***
- XPACK.MANAGEMENT.ELASTICSEARCH.SSL.CERTIFICATE_AUTHORITY=/usr/share/logstash/certs/ca/ca.crt
- XPACK.MANAGEMENT.ELASTICSEARCH.SSL.VERIFICATION_MODE=certificate
- XPACK.MANAGEMENT.PIPELINE.ID=["test_cpm", "test", "beats"]
- XPACK.MANAGEMENT.LOGSTASH.POLL_INTERVAL=10s

```

I have already tried to use _logstash\_user_ for both, management and monitoring by granting

> cluster:admin/xpack/monitoring/bulk

to _logstash\_writer_ role. Also tried _logstash\_system_ as "run as" for _logstash\_user_. no success so far... Any hints?

It seems to me, that using different credentials for monitoring and management pointing to the very same elastic instances leads to this error - is this plausible? Can logstash handle just one user/pass for one ES-Connection?

Cheers,  
Rafael

BTW: I´ve followed the docs closely, and everything is working but the described "monitor vs. management" collision... I can have one or another, but not both at the same time...

---

<div class="post-metadata">

**Author:** ![Mike\_Place](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mike_place/32/39555_2.png) [@Mike\_Place](https://discuss.elastic.co/u/Mike_Place)\
**Post date:** [November 26, 2019, 11:38am UTC](https://discuss.elastic.co/t/logstash-monitoring-vs-management/209387/2 "2019-11-26T11:38:31Z")

</div>

Hi @ibexit. I see a similar comment in a GitHub issue filed recently:

> <https://github.com/elastic/logstash/issues/9126#issuecomment-557671292>
>
> During LS startup I can see following message multiple times:
> \[logstash.inputs.metrics \] Failed to create monitoring event {:message=\>"undefined method \`ephemeral\_id' for nil:NilClass",...

Perhaps this is a bug affecting others? (Or, perhaps you filed this?)

---

<div class="post-metadata">

**Author:** ![ibexit](https://avatars.discourse-cdn.com/v4/letter/i/a88e4f/32.png) [@ibexit](https://discuss.elastic.co/u/ibexit)\
**Post date:** [November 26, 2019, 6:41pm UTC](https://discuss.elastic.co/t/logstash-monitoring-vs-management/209387/3 "2019-11-26T18:41:18Z")

</div>

Hi @Mike_Place,

thank you for the fast response. I´ll link the logs produced in the three scenarios.

**1. monitoring only:**  
(pastebin logs: [monitoring works](https://pastebin.com/8x4WE8qK))

**2. pipeline mannagement only:**  
(pastebin logs: [pipeline management works](https://pastebin.com/yDgZu7H7))

**3. both, management and monitoring:**  
(pastebin logs: [pipeline management works, no monitoring data](https://pastebin.com/kVdh9LjJ))  
Additionaly, after resetting the logstash\_writer role (removed cluster:admin/xpack/monitoring/bulk cluster permission and removed _logstash\_system_ from "run as") to the [elastic reference docs state](https://www.elastic.co/guide/en/logstash/current/ls-security.html) this error appears in this scenario:

```
logstash | [2019-11-26T19:27:46,237][ERROR][logstash.outputs.elasticsearch] Encountered a retryable error. Will Retry with exponential backoff {:code=>403, :url=>"https://192.168.6.17:9250/_monitoring/bulk?system_id=logstash&system_api_version=7&interval=1s"}

```

But I can't find any error similar to the one you've referenced/mentioned.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 24, 2019, 6:41pm UTC](https://discuss.elastic.co/t/logstash-monitoring-vs-management/209387/4 "2019-12-24T18:41:18Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.