Clean install and running as a service: no files in /etc/logstash/conf.d/*

Elastic Stack Logstash
1

Hi,

I am setting up logstash 5.5.1 in a RHEL 7 and my java version is 1.8.0_144 (Oracle).

I have been following the tutorials and it seems that the pipeline configuration files are placed in the /etc/logstash/conf.d directory. I have installed Logstash using the RPM and doing "yum -y localinstall logstash-5.5.1.rpm".

I have enabled the service using systemctl and It is running, however the log is reporting the following error: No config files found in path {:path=>"/etc/logstash/conf.d/*"}

This is happening because the conf.d directory is empty. So I tried to create some links in /etc/logstash/conf.d/ pointing to the configuration files stored in /etc/logstash/: jvm.options, log4j2.properties, logstash.yml, and startup.options.

If I do that, there is a new error:

[2017-08-22T15:38:22,078][ERROR][logstash.agent ] Cannot create pipeline {:reason=>"Expected one of #, input, filter, output at line 6, column 1 (byte 132) after ## JVM configuration\n\n# Xms represents the initial size of total heap space\n# Xmx represents the maximum size of total heap space\n\n"}

I am running out of ideas, how could I set up the configuration files properly?

Anyays, I have some questions:

  • Why is Logstash taking the configuration files from conf.d instead of /etc/logstash/ (even if it is a service, why is not using those?)

  • Is Logstash generating a mix of all the configuration in a single configuration file? Why is doing that?

  • I am following this tutorial: https://www.elastic.co/guide/en/logstash/5.5/installing-logstash.html , am I missing any other tutorial?

2

create an empty file logstash.conf in /etc/logstash/conf.d and run logstash.

2 Likes
3

Hello,

Vishal_Sharma1's solution is the quick one :slight_smile:

Your answers :

  1. By default (Unix Deb), Logstash core conf is in /etct/logstash/ and logstash pipeline (what you do with processed data) conf is in /etc/logstash/conf.d. Like a web server (nginx/apache), you have the web server settings itself and the virtual host conf for one or more websites.
  2. When you start Logstash process/service, it merge all pipeline conf files (conf.d/) into one file ; so be aware to use a single file or alphabetical filenames to be sure that logstash read your pipeline conf in the correct order. I don't know the reason....
  3. It's the correct doc, but you seem confusing between core and pipeline confs.
1 Like
5

When you start Logstash process/service, it merge all pipeline conf files (conf.d/) into one file ; so be aware to use a single file or alphabetical filenames to be sure that logstash read your pipeline conf in the correct order. I don’t know the reason…

How did you expect things to work?

6

Hello Magnus,

I didn't expect anything, it simple and efficient for me, but there is maybe other best behaviour, I'm curious :slight_smile:

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.