Logstash.conf not found for after installing logstash

Elastic Stack Logstash
1

I have installed the logstash with the below link but I am not able to find logstash.conf in my system at the paths

/etc/logstash/cont.d
/etc/init/
/etc/init.d/

https://www.elastic.co/guide/en/logstash/current/installing-logstash.html

The OS is Ubuntu 16.04

2

What's inside of /etc/logstash?

3

I find below files

sawyer@algo-11:/etc/logstash$ ls
conf.d jvm.options log4j2.properties logstash.yml startup.options

sawyer@algo-11:/etc/logstash$ cd conf.d
sawyer@algo-11:/etc/logstash/conf.d$ ls -lrt
total 0

When I provide the below commands it gives

sawyer@algo-11:~$ sudo systemctl start logstash.service
sawyer@algo-11:~$ sudo initctl start logstash
initctl: Unable to connect to Upstart: Failed to connect to socket /com/ubuntu/upstart: Connection refused

I have also run the command before installation

nbhagwath@algo-11: export JAVA_HOME=/opt/jdk1.8.0_151

4

Logstash doesn't ship with a file named logstash.conf. Are you conflating that file with logstash.yml (which should be found in /etc/logstash) or what makes you think logstash.conf should exist anywhere?

1 Like
5

Fine now after going through the tutorials I was able to find it. But now I have given the encoder, inside the logback.xml. Only the debug and error logs are turned into JSON format, and other info, trace or not.

6

So you do have a conf.d, you just don't have anything in it. You need to
create a file in it that has an input, filter and output inside. Then try
running things

7

Hi, I have created.

Can you please let me know how do I capture all the types of the Logger.(ERROR, DEBUG, TRACE, INFO, and WARN)

8

try debug or trace

9

where should I try and configure the same

10

where should I try and configure the same

Does the log file produced by your (Java?) application contain messages of all loglevels? If not then this isn't a Logstash problem.

11

/etc/logstash/logstash.yml. at the bottom

13

.I don't understand what you're asking

14

@seanziee: I have edited my question

15

@magnusbaeck: please answer to this question. It is the four questions above that I am facing available in this thread

17

please answer to this question. It is the four questions above that I am facing available in this thread

Then please answer this question: Does the log file produced by your (Java?) application contain messages of all loglevels?

18

Yes it has all the log levels

19

Unless you have configured Logstash to drop certain messages I find it hard to believe that some messages aren't being processed. What does your Logstash configuration look like? And have you tried creating a minimal example that reproduces the problem? Like a tiny file with just a few log entries and a stdout { codec => rubydebug } Logstash output that just dumps all events?

20

Thanks for the acknowledgement,

I have created a file named logstash.conf inside /etc/logstash/conf.d/logstash.conf. The configuration is as below. But when with the below command. Even though I have added the line path.settings: /etc/logstash/conf.d/ inside logstash.conf I still get the same issue.

  **sawyer@ford11:/usr/share/logstash$ bin/logstash -f logstash.conf**
  WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or 
  /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
  Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using 
  default config which logs errors to the console
 [ERROR] 2018-01-29 11:27:50.728 [Ruby-0-Thread-1: 
  /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22] 
  sourceloader - No configuration found in the configured sources.


   input { stdin { } } 
   filter {
      mutate {
          strip => "message"
         }
     }
    output { 
            stdout { } 
            }
21

There are several problems here.

  • You're running bin/logstash -f logstash.conf from /usr/share/logstash but logstash.conf is stored in /etc/logstash/conf.d so the file obviously won't be found.
  • path.settings isn't a valid Logstash setting (see https://www.elastic.co/guide/en/logstash/current/logstash-settings-file.html).
  • Even if path.settings were valid it should go in the settings file, logstash.yml, not a pipeline configuration file stored in /etc/logstash/conf.d.
22

Please let me know where do I run
bin/logstash -f logstash.conf

when I run as below I get the error as
saywer@ford11:/etc/logstash/conf.d$ logstash -f logstash.conf
logstash: command not found