In RHEL how to run Configuration file along with logstash?

so according that documentation, we need to create different pipelines with unique id and in the path we need to specify which config file to select , right??

Yes.

what is that workers means??

Have you looked in the documentation?

thanks @magnusbaeck, i hope problem will resolve now.

workers come into picture if we consider cpu utilization i think according to documentation??!

Hi @magnusbaeck

In RHEL environment we dont have pipelines.yml file to make changes to create multiple pipelines.

so i did configuration in Logstash.yml file as follows:

#------------ Pipeline Settings --------------

  • pipeline.id: monerrorlog-pipeline
    path.config: "/etc/logstash/conf.d/mon-errorlog.conf"
    pipeline.workers: 1
  • pipeline.id: monserverlog-pipeline
    path.config: "/etc/logstash/conf.d/mon-serverlogs.conf"
    pipeline.workers: 1
  • pipeline.id: sl1a-pipeline
    path.config: "/etc/logstash/conf.d/sl1a-serverlogs.conf"
    pipeline.workers: 1
  • pipeline.id: sl2a-pipeline
    path.config: "/etc/logstash/conf.d/sl2a-serverlogs.conf"
    pipeline.workers: 1
  • pipeline.id: sl2b-pipeline
    path.config: "/etc/logstash/conf.d/sl2b-serverlogs.conf"
    pipeline.workers: 1
  • pipeline.id: sl2c-pipeline
    path.config: "/etc/logstash/conf.d/sl2c-serverlogs.conf"
    pipeline.workers: 1
  • pipeline.id: sl3a-pipeline
    path.config: "/etc/logstash/conf.d/sl3a-serverlogs.conf"
    pipeline.workers: 1
  • pipeline.id: sl3b-pipeline
    path.config: "/etc/logstash/conf.d/sl3b-serverlogs.conf"
    pipeline.workers: 1
  • pipeline.id: sl3c-pipeline
    path.config: "/etc/logstash/conf.d/sl3c-serverlogs.conf"
    pipeline.workers: 1
  • pipeline.id: sl3d-pipeline
    path.config: "/etc/logstash/conf.d/sl3d-serverlogs.conf"
    pipeline.workers: 1

#------------ Pipeline Configuration Settings --------------
#Where to fetch the pipeline configuration for the main pipeline

#path.config: /etc/logstash/conf.d/*.conf

#Pipeline configuration string for the main pipeline

#config.string:

#At startup, test if the configuration is valid and exit (dry run)

#config.test_and_exit: false

#Periodically check if the configuration has changed and reload the pipeline
#This can also be triggered manually through the SIGHUP signal

#config.reload.automatic: false

#How often to check if the pipeline configuration has changed (in seconds)

#config.reload.interval: 3s

#Show fully compiled configuration as debug log message
#NOTE: --log.level must be 'debug'

#config.debug: false

#When enabled, process escaped characters such as \n and " in strings in the
#pipeline configuration files.

#config.support_escapes: false
.....

so am i done configuration correct??

let me know if any changes required!

Thanks,
Naveena K N

Which version of Logstash have you installed?

hi @Christian_Dahlqvist

its Logstash 6.0.0 in RHEL

Since 6.x they introduced pipelines configuration. You should be able to see pipelines.yml file in your logstash installation folder. Anyway let me download and see.

ya suman...please let me know aftr installing
Thanks,
Naveena K N

If it is not there try to create pipelines.yml file in you config folder and start logstash with bin/logstash -r option

its in RHEL operating system , so are you sure we can start using bin/logstash -r??

Well that am not sure of. Ours is Linux - GNU/Linux. Try to start it in your local machine. Then you can try it in RHEL or anywhere. Or your local itself is RHEL? :stuck_out_tongue:

ok will wait for ELK team Response

its in RHEL operating system , so are you sure we can start using bin/logstash -r??

The choice of Linux distribution doesn't affect whether the -r option works.

pipelines.yml file is not there.!!

shall i add pipelines.yml??

Thanks,
Naveena K N

try adding it. There is nothing wrong in trying right.!

yes suman, we asked team who are have permissions to upload file in the current working environment...

Thanks,
Naveena

What I would suggest or prefered to do is.. Do the testing and all the stuff in your local machine and if the configuration works well go to the higher environment. That way you can save lot of time. Just think about it.

yes iam testing it in local also with pipeline.yml having :

pipeline.id: test
pipeline.workers: 2
pipeline.batch.size: 1
path.config: "C:\Users\nakn\Downloads\logstash-6.1.1\bin\config\logs-server.conf"

but when i tried to run logstash as follows:

bin/logstash

im getting error as Failed to read pipelines yaml file !!
why am i getting this error??

Thanks,
Naveena

Can we have detailed fail log?

ya sure

I dont see any error here