Pipelines.yml RemoteConfigError - Where should pipeline configs reside?

We're trying to use the multiple pipelines feature described here in Logstash/ES 6.5 on Windows but LogStash can apparently not see/find/read the .conf files.

We're starting logstash.bat with no parameters and our pipelines.yml looks like this:

# Pipeline for FileBeats: Application console and access logs
- pipeline.id: filebeat
  path.config: "/elk/logstash/config/pipeline.filebeat.conf"

# Pipeline for other application logs
- pipeline.id: other.logs
  path.config: "pipeline.other.logs.conf"

The error is:

[2018-12-18T10:43:43,542][ERROR][logstash.config.sourceloader] Could not fetch all the sources {:exception=>LogStash::ConfigManagement::ElasticsearchSource::RemoteConfigError, :message=>"Empty configuration for pipeline_id: filebeat", :backtrace=>["D:/elk/logstash/vendor/bundle/jruby/2.3.0/gems/x-pack-6.2.2-java/lib/config_management/elasticsearch_source.rb:101:
in `get_pipeline'", "D:/elk/logstash/vendor/bundle/jruby/2.3.0/gems/x-pack-6.2.2-java/lib/config_management/elasticsearch_source.rb:87:

We have tried every variation of path.config: "pipeline.filebeat.conf" or path.config: "config\pipeline.filebeat.conf" etc.

It seems LogStash cannot find these .conf files but they do indeed exist in the config folder.

Adding --debug has not helped in seeing where LogStash expects these files.

BONUS QUESTION: How do the pipelines defined in pipeline.yml relate to those defined in logstash.yml in xpack.management.pipeline.id and those created in Kibana??

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.