Info message "No config files found in path" - doesn't make any sense

[dorj1234 (Dor Juravski)]

Hi,
I have been using 2 files for a long time
/path/to/logstash1.conf
/path/to/logstash2.conf

When switching to pipelines.yml I get an error that tells me no config files found in path (all correct path and file names)
[2018-01-02T14:20:19,349][INFO ][logstash.config.source.local.configpathloader] No config files found in path {:path=>"/path/to/logstash1.conf"}
[2018-01-02T14:20:19,351][INFO ][logstash.config.source.local.configpathloader] No config files found in path {:path=>"/path/to/logstash2.conf"}

This is wrong because:

1. The files and path to the files are accessible and readable to logstash, to everyone
2. The files are correct. They work when I use manual logstash (logstash -f /path/to/logstash1.conf ...)

This doesn't make any sense.
Any advice is much appreciated.
Using recent version 6.x

Thanks in advance !
JD

[guyboertje]

Do you have path.config set in logstash.yml AND pipelines defined in pipelines.yml?

[dorj1234 (Dor Juravski)]

No path.config in logstash.yml since it contradicts using a service. If I do that, my pipelines.yml will be ignored

As for pipelines.yml - yes it's well defined

[dorj1234 (Dor Juravski)]

Just gave it another try.
When path.config is defined in /etc/logstash.yml --> the logs says it ignores pipelines.yml
When path.config is not defined in /etc/logstash.yml --> pipelines.yml is being read BUT I get the "no config files found in path" when this does not make any sense.

The config files in path are the 2 conf files I have been using all the time. They work, they are valid, they are readable.

This mechanism never worked, I am not doing anything complex here, just trying to run a logstash conf file via pipelines.yml

[guyboertje]

Please post your pipelines.yml file contents. Use Markdown triple backticks above and below what you paste.

[dorj1234 (Dor Juravski)]

Sure here it is:
Those were .conf files originally, I thought maybe it's really looking for file extension (.config or .cfg)

- pipeline.id: testunified
  path.config: "/home/elastic/unified/test/conf/v2_unified.conf"
- pipeline.id: test2
  path.config: "/home/elastic/testfiles/test2/test2_vpc/conf/v4_logstash.conf

Everything works when I call them individually with (for example)
logstash -f /path_to.../v2_unified.conf --path.settings /etc/logstash

My /etc/logstash/logstash.yml is mostly commented out. The only lines in there are:

node.name: elk1
path.data: /var/lib/logstash
log.level: warn
path.logs: /var/log/logstash

Note that if I un-comment path.config: /etc/logstash/conf.d then logstash ignores pipelines.yml in my case. My guess is it's reverting to logstash.yml because it can't handle the pipelines.yml setting.

Thank you!
JD

[guyboertje]

You are correct, that does not make sense.

Have you tried moving those two conf files into, say, /etc/logstash/testing/?