[SOLVED] Trying monitoring logstash in kibana/elasticsearch

Elastic Stack Logstash
1

Hi again forum!

I've been again playing with ELK stuff these days, I've upgraded my 3 VM baby cluster lab from 5.4 to 6.3, and started adding new data onto it ... and I've realized that it is possible to monitor logstash in kibana, alongside Elasticsearch and Kibana itself ... I find all this fantastic, but I miss logstash being monitored by default.

I've read the documentation... and it lloked simple enough to dare trying it ... but I failed all the time.

If I add this pair of lines at the end of /etc/logstash/logstash.yml

xpack.monitoring.elasticsearch.url:
["http://192.168.1.4:9200", "http://192.168.1.5:9200", "http://192.168.1.6:9200"]

logstash starts ... but it says on logs that it can't connect to elasticsearch, and, it never raises the filebeat listening services... the process starts, but never ends the startup.

So, I though I was missing something on elasticsearch side, and so, I added the following line at the end of my mostly default elasticsearch.yml

xpack.monitoring.collection.enabled: true

Which causes elasticsearch to process start, but to not join the cluster or become really 'up'

My setup has no xpack stuff setup, no SSL, no authentication... mostly default package install.
By reading the docs, I've no clue on how to go next ... it just 'don't work'

... could someone give me some advice, some clue, or some link to a 'howto' on how set this up?

Thank you very much in advance, best regards.xpack.monitoring.enabled: "true"

EDIT / solution (just doesn't crash and show on Kibana... don't tested to which extent does this work):

  • Logstash side .yml config:

xpack.monitoring.enabled: "true"
xpack.monitoring.elasticsearch.url: ["http://192.168.1.4:9200", "http://192.168.1.5:9200", "http://192.168.1.6:9200"]

  • Elasticsearch side:

ADD NOTHING / Leave as package default

2

xpack with newest elasticsearch comes already with binary, you don't have to install it.
I don't know if that changes much, but have you activated basic license for your new installation?

POST _xpack/license/start_trial --> 30d trial (with all features)
POST _xpack/license/start_basic --> to start basic license
GET _xpack/license/basic_status --> to see if you are entitled to run basic license

3

Hi! thank you for your reply.

It seems it is working ... so, as of 6.3, everything comes ready almost 'out of the box' (I just missed to interpret from the docs to add the line ' xpack.monitoring.enabled: "true"' , and that nothing had to be done on elasticsearch part)

Out of curiosity, I executed your GET HTTP request (I've seen that kibana now integrates a tool to struggle with using those weird API requests :-P)
The answer was:

{
"eligible_to_start_basic": false
}

Which I don't know what it means, but anyhow, it allows or at least doesn't prevent the functuonality to work (as it is working :-D)

4

Hmm, you are not eligible.
I wonder if that is connected with the upgrade from major release name.

Logstash 6.4 comes with pre-commeted settings in /etc/logstash/logstash.yml

X-Pack Monitoring

Monitoring Logstash with APIs | Logstash Reference [8.11] | Elastic

xpack.monitoring.enabled: true
#xpack.monitoring.elasticsearch.username: logstash_system
#xpack.monitoring.elasticsearch.password: password
xpack.monitoring.elasticsearch.url: ["http://your_elasticsearch:9200", "http://your_elasticsearch:9200"]
#xpack.monitoring.elasticsearch.ssl.ca: [ "/path/to/ca.crt" ]
#xpack.monitoring.elasticsearch.ssl.truststore.path: path/to/file
#xpack.monitoring.elasticsearch.ssl.truststore.password: password
#xpack.monitoring.elasticsearch.ssl.keystore.path: /path/to/file
#xpack.monitoring.elasticsearch.ssl.keystore.password: password
#xpack.monitoring.elasticsearch.ssl.verification_mode: certificate
#xpack.monitoring.elasticsearch.sniffing: false
#xpack.monitoring.collection.interval: 10s
#xpack.monitoring.collection.pipeline.details.enabled: true

Uncomment those two lines, fill the es information and done .
It does not need any Kibana reboot to see it under Monitoring section.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.