Logstash error : invalid setting for elasticsearch output plugin

Elastic Stack Logstash
1

Hi,

I have two pipeline configurations located in /etc/logstash/conf.d. I can make work both seperately by running /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/**.conf -r, and it sends data to elasticsearch as I want.

The output is the same for both except the index :

output {
  stdout { }
  elasticsearch {
    hosts => ["https://myip:9200"]
    index => "switch-%{+YYYY.MM.dd}"
    ssl => true
    ssl_certificate_verification => true
    cacert => "/etc/logstash/certs/http_ca.crt"
    user => "${es_user}"
    password => "${es_pwd}"
  }
}

So I tried to make the two configurations work together. Here's my pipeline.yml :

- pipeline.id: switch
  path.config: "/etc/logstash/conf.d/switch.conf"
- pipeline.id: esxi
  path.config: "/etc/logstash/conf.d/esxi.conf"

And here's the error I get when I try to run logstash (with systemctl start logstash) :

sept. 12 14:57:47 local logstash[45016]: [2022-09-12T14:57:47,421][ERROR][logstash.outputs.elasticsearch] Invalid setting for elasticsearch output plugin:
sept. 12 14:57:47 local logstash[45016]:   output {
sept. 12 14:57:47 local logstash[45016]:     elasticsearch {
sept. 12 14:57:47 local logstash[45016]:       # This setting must be a path
sept. 12 14:57:47 local logstash[45016]:       # File does not exist or cannot be opened /etc/logstash/certs/http_ca.crt
sept. 12 14:57:47 local logstash[45016]:       cacert => "/etc/logstash/certs/http_ca.crt"
sept. 12 14:57:47 local logstash[45016]:       ...
sept. 12 14:57:47 local logstash[45016]:     }
sept. 12 14:57:47 local logstash[45016]:   }
sept. 12 14:57:47 local logstash[45016]: [2022-09-12T14:57:47,425][ERROR][logstash.agent           ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:esxi-pipeline, :exception=>"Java::JavaLang::IllegalStateException", :message=>"Unable to configure plugins: (ConfigurationError) Something is wrong with your configuration.", :backtrace=>["org.logstash.config.ir.CompiledPipeline.<init>(CompiledPipeline.java:120)", "org.logstash.execution.JavaBasePipelineExt.initialize(JavaBasePipelineExt.java:85)", "org.logstash.execution.JavaBasePipelineExt$INVOKER$i$1$0$initialize.call(JavaBasePipelineExt$INVOKER$i$1$0$initialize.gen)", "org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:846)", "org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuper(IRRuntimeHelpers.java:1229)", "org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuperSplatArgs(IRRuntimeHelpers.java:1202)", "org.jruby.ir.targets.indy.InstanceSuperInvokeSite.invoke(InstanceSuperInvokeSite.java:29)", "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$initialize$0(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:48)", "usr.share.logstash.logstash_minus_core.lib.logstash.pipeline_action.create.RUBY$method$execute$0(/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:50)", "usr.share.logstash.logstash_minus_core.lib.logstash.agent.RUBY$block$converge_state$2(/usr/share/logstash/logstash-core/lib/logstash/agent.rb:381)", "org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:141)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:64)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:58)", "org.jruby.runtime.Block.call(Block.java:143)", "org.jruby.RubyProc.call(RubyProc.java:309)", "org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:107)", "java.base/java.lang.Thread.run(Thread.java:833)"]}

I tried to modify the path with ', [ but it didn't change anything.

I found other topics with other plugins not related to elasticsearch in particular so I guess it is more of a Logstash probem than an Elasticsearch one.

Thanks for your help !

2

How are you running logstash? As a service? Did you check the permissions of this path?

3

Hi, thanks for your quick answer !

Yes logstash is running as a service.

Regarding the permissions, I changed the owner of the file http_ca.crt from root to logstash but it didn't change anything. But while answering, I changed the owner of the whole path and it seems to have done something, I don't have this error anymore. It is not working yet as I have another issue but I will keep you posted if the problem is back.

I ran chown logstash:logstash /etc/logstash/certs for those that it could help.

Thanks so much !!

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.