Unable to monitor logstash from kibana

Hi Team,

I am trying to monitor logstash from Kibana. Ihave modified the logstash.yml file but its not adding to kibana / not able to see it from kibana.

My Elasticsearch is running with TLS.

added below parameters in logstash.yml file.

# X-Pack Monitoring
# https://www.elastic.co/guide/en/logstash/current/monitoring-logstash.html
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.username: logstash_system
xpack.monitoring.elasticsearch.password: XXXXXXXX
#xpack.monitoring.elasticsearch.proxy: ["https://XX.XX.XX.XX:9200"]
xpack.monitoring.elasticsearch.hosts: ["https://XX.XX.XX.XX:9200", "https://XX.XX.XX.XX:9200","https://XX.XX.XX.XX:9200"]
# an alternative to hosts + username/password settings is to use cloud_id/cloud_auth
#xpack.monitoring.elasticsearch.cloud_id: monitoring_cluster_id:xxxxxxxxxx
#xpack.monitoring.elasticsearch.cloud_auth: logstash_system:password
# another authentication alternative is to use an Elasticsearch API key
#xpack.monitoring.elasticsearch.api_key: "id:api_key"
xpack.monitoring.elasticsearch.ssl.certificate_authority: [ "/etc/logstash/elk-stack.crt.pem" ]
xpack.monitoring.elasticsearch.ssl.truststore.path: /etc/logsatsh/cert.jks
xpack.monitoring.elasticsearch.ssl.truststore.password: password
#xpack.monitoring.elasticsearch.ssl.keystore.path: /path/to/file

i am getting below error if i run the systemctl start logstash command

systemctl status is showing running ..

[root@lnitiblbk2u logstash]# systemctl status logstash
● logstash.service - logstash
   Loaded: loaded (/usr/lib/systemd/system/logstash.service; disabled; vendor preset: disabled)
   Active: active (running) since Fri 2022-07-15 13:32:36 +04; 447ms ago
 Main PID: 24990 (java)
   CGroup: /system.slice/logstash.service
           └─24990 /usr/share/logstash/jdk/bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djruby.compile.invokedy...

Jul 15 13:32:36 lnitiblbk2u systemd[1]: Started logstash.
Jul 15 13:32:36 lnitiblbk2u logstash[24990]: Using bundled JDK: /usr/share/logstash/jdk
Jul 15 13:32:36 lnitiblbk2u logstash[24990]: OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.

logstash logs -

[2022-07-15T13:33:13,338][INFO ][logstash.runner          ] Log4j configuration path used is: /etc/logstash/log4j2.properties
[2022-07-15T13:33:13,401][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"8.3.2", "jruby.version"=>"jruby 9.2.20.1 (2.5.8) 2021-11-30 2a2962fbd1 OpenJDK 64-Bit Server VM 11.0.15+10 on 11.0.15+10 +indy +jit [linux-x86_64]"}
[2022-07-15T13:33:13,403][INFO ][logstash.runner          ] JVM bootstrap flags: [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-opens=java.base/java.security=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.nio.channels=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.management=ALL-UNNAMED]
[2022-07-15T13:33:13,417][FATAL][org.logstash.Logstash    ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby.jar:?]
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby.jar:?]
        at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:91) ~[?:?]

but its not added to kibana.

You have a FATAL error when starting Logstash, your Logstash is not running. Do you have anything else in the logs after that? Check /var/log/messages or /var/log/syslog and search for logstash to see if you have some hint of why logstash didn't start.

Also, share your logstash.yml file.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.