Hello,
Been struggling with this issue for a few days. Not sure what's going on. Here's the command executed:
root@machine:/home/devops# /usr/share/logstash/bin/logstash --config.debug --log.level=debug --path.settings /etc/logstash --path.logs /var/log/logstash
And the log messages from the command execution:
Using bundled JDK: /usr/share/logstash/jdk
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
Sending Logstash logs to /var/log/logstash which is now configured via log4j2.properties
[2022-08-03T15:45:30,358][INFO ][logstash.runner ] Log4j configuration path used is: /etc/logstash/log4j2.properties
[2022-08-03T15:45:30,372][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"8.3.3", "jruby.version"=>"jruby 9.2.20.1 (2.5.8) 2021-11-30 2a2962fbd1 OpenJDK 64-Bit Server VM 11.0.15+10 on 11.0.15+10 +indy +jit [linux-x86_64]"}
[2022-08-03T15:45:30,376][INFO ][logstash.runner ] JVM bootstrap flags: [-Xms512m, -Xmx512m, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-opens=java.base/java.security=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.nio.channels=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.management=ALL-UNNAMED]
[2022-08-03T15:45:30,389][DEBUG][logstash.modules.scaffold] Found module {:module_name=>"fb_apache", :directory=>"/usr/share/logstash/modules/fb_apache/configuration"}
[2022-08-03T15:45:30,392][DEBUG][logstash.plugins.registry] Adding plugin to the registry {:name=>"fb_apache", :type=>:modules, :class=>#<LogStash::Modules::Scaffold:0x433f9449 @directory="/usr/share/logstash/modules/fb_apache/configuration", @module_name="fb_apache", @kibana_version_parts=["6", "0", "0"]>}
[2022-08-03T15:45:30,393][DEBUG][logstash.modules.scaffold] Found module {:module_name=>"netflow", :directory=>"/usr/share/logstash/modules/netflow/configuration"}
[2022-08-03T15:45:30,394][DEBUG][logstash.plugins.registry] Adding plugin to the registry {:name=>"netflow", :type=>:modules, :class=>#<LogStash::Modules::Scaffold:0x114f913d @directory="/usr/share/logstash/modules/netflow/configuration", @module_name="netflow", @kibana_version_parts=["6", "0", "0"]>}
Your settings are invalid. Reason: Cloud Id, after decoding, is invalid. Format: '<segment1>$<segment2>$<segment3>'. Received: "".
[2022-08-03T15:45:30,423][FATAL][org.logstash.Logstash ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby.jar:?]
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby.jar:?]
at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:91) ~[?:?]
Note: changed the Java virtual machine limits because the machine has 1gb of memory.
logstash.yml (focused on cloud ID section)
# Define Elastic Cloud settings here.
# Format of cloud.id is a base64 value e.g. dXMtZWFzdC0xLmF3cy5mb3VuZC5pbyRub3RhcmVhbCRpZGVudGlmaWVy
# and it may have an label prefix e.g. staging:dXMtZ...
# This will overwrite 'var.elasticsearch.hosts' and 'var.kibana.host'
cloud.id: 'elastic-test:%Cloud ID is here%
# Format of cloud.auth is: <user>:<pass>
# This is optional
# If supplied this will overwrite 'var.elasticsearch.username' and 'var.elasticsearch.password'
# If supplied this will overwrite 'var.kibana.username' and 'var.kibana.password'
# cloud.auth: elastic:<password>
cloud.auth: 'logstash:%Password is here%
pipelines.yml
"/etc/logstash/pipelines.yml" 6L, 285C 6,1 All
# This file is where you define your pipelines. You can define multiple.
# For more information on multiple pipelines, see the documentation:
# https://www.elastic.co/guide/en/logstash/current/multiple-pipelines.html
- pipeline.id: main
path.config: "/etc/logstash/conf.d/*.conf"
Pipeline configuration is a single system linked file
root@machine:/home/devops# ls -l /etc/logstash/conf.d/
total 0
lrwxrwxrwx 1 root root 41 Aug 2 20:15 provider_model.conf -> /home/devops/logstash/provider_model.conf
Part of the pipeline config file:
output {
stdout { codec => rubydebug }
elasticsearch {
hosts => "https://machine.elastic-cloud.com:9243"
user => "logstash"
password => "%Password is here%"
index => "provider-%{+YYYY.MM.03}"
manage_template => true
template => "/home/devops/logstash/provider_model_mapping_template.json"
template_name => "provider_template"
template_overwrite => true
}
}
Note: the commands show root as the current user, but the commands have also been executed with the devops user. The result is identical.
What is really confusing is this log line, and I'm assuming is the reason for failure?
Your settings are invalid. Reason: Cloud Id, after decoding, is invalid. Format: '<segment1>$<segment2>$<segment3>'. Received: "".