Multiple Logstash pipelines

(Diaconita Costinel) #1

Hi,

I am using Logstash 6.5 as a service on a windows machine. I have defined a pipelines.yml file that looks like this:
# List of pipelines to be loaded by Logstash
- pipeline.id: filebeat
path.config: "CPA/LogStash/config\logstash-filebeat.conf"

# List of pipelines to be loaded by Logstash
 - pipeline.id: K5DLWDBSSIT2909
   path.config: "CPA/LogStash/config/logstash-K5DLWDBSSIT2909.conf"

When i run logstash from cmd whitout a parameter like '-f' or '-e' the output is like thiis:
"Cannot read the pipelines file"

Does anyone have any ideea how can i resolve this?

Thanks!

#2

Is your pipelines.yml file in the directory you have set in path.settings?

(Diaconita Costinel) #3

path.settings is located in logstash.yml?
Pipelines.yml is located in the same folder with logstash.yml

#4

Unless you override it on the command line, paths.settings would be set in logstash.yml, yes.

(Diaconita Costinel) #5

The config files that i defined in the path.config must be in the same place where logstash is installed? I don't override in the command line and is still doesn't work..

(Diaconita Costinel) #6

The error is now "Failed to read pipeline yaml file" It matters on which drive is logstash installed? Does it have to be C drive?

#7

Can you post the entire output from logstash when you run it from the command line. Please post it as text, select it, and click on </> in the toolbar above the edit panel. You should then see it displayed in the preview panel on the right like this

...
(Diaconita Costinel) #8

C:\Users\zDL-ChERP_GSTS_setup>E:\TotalSoft\CPA\LogStash\bin\logstash.bat
Sending Logstash logs to K:/LogStash/Log which is now configured via log4j2.prop
erties
ERROR: Failed to read pipelines yaml file. Location: E:/TotalSoft/CPA/LogStash/c
onfig/pipelines.yml
usage:
bin/logstash -f CONFIG_PATH [-t] [-r] [-w COUNT] [-l LOG]
bin/logstash --modules MODULE_NAME [-M "MODULE_NAME.var.PLUGIN_TYPE.PLUGIN_NAM
E.VARIABLE_NAME=VALUE"] [-t] [-w COUNT] [-l LOG]
bin/logstash -e CONFIG_STR [-t] [--log.level fatal|error|warn|info|debug|trace
] [-w COUNT] [-l LOG]
bin/logstash -i SHELL [--log.level fatal|error|warn|info|debug|trace]
bin/logstash -V [--log.level fatal|error|warn|info|debug|trace]
bin/logstash --help
[2019-05-16T18:04:17,952][ERROR][org.logstash.Logstash ] java.lang.IllegalSta
teException: Logstash stopped processing because of an error: (SystemExit) exit

#9

Does that file exist?

(Diaconita Costinel) #10

Yes!

#11

If you cd to the E: drive and run the same command do you get the same error?

(Diaconita Costinel) #12

Yes, i get the same error. I don't know what else to try.

(Pascal GUINET) #13

Since it seems to be on Windows, put backslashes "" and not slashes ? (just guessing, never used Elasticsearch on Windows).

(Diaconita Costinel) #14

Hi,

Now i get a different error : error logstash.config.sourceloader no configuration found in the configured sources.

Any help?
Thanks!