Logstash Configuration Issues

[Bheeshma]

Hi,
I am new to this forum, trying to learn the ELK stack on my own . Thanks to ELK documentation on the website uploaded.

i was able to install and configure Elastic Search, Kibana softwares however when trying to configure Logstash, I am stuck at this and need your help on how to overcome this please.

After installing logstash, I configured *.conf with the following code and tried to run the command : ./logstash but it is giving lots of errors.

logstash.confg

input { beats {port => 5044 } }

output {
elasticsearch { hosts => ["localhost:9200"] }
stdout { codec => rubydebug }
}

Errors:

Sending Logstash's logs to /home/oracle/logstash-6.1.1/logs which is now configured via log4j2.properties
2018-01-18 08:21:07,378 main ERROR Unable to invoke factory method in class class org.apache.logging.log4j.core.layout.JsonLayout for element JSONLayout. java.lang.reflect.InvocationTargetException

Caused by: java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.ser.impl.SimpleFilterProvider.addFilter(Ljava/lang/String;Lcom/fasterxml/jackson/databind/ser/impl/SimpleBeanPropertyFilter;)

Lcom/fasterxml/jackson/databind/ser/impl/SimpleFilterProvider;

2018-01-18 08:21:07,408 main ERROR Unable to invoke factory method in class class org.apache.logging.log4j.core.layout.JsonLayout for element JSONLayout. java.lang.reflect.InvocationTargetException

[2018-01-18T08:21:09,957][ERROR][logstash.agent ] An exception happened when converging configuration {:exception=>RuntimeError, :message=>"Could not fetch the configuration, message: Failed to coerce value to Integer.

Received 1 (actually defaults to number of CPUs) (String)", :backtrace=>["/home/oracle/logstash-6.1.1/logstash-core/lib/logstash/agent.rb:155:in `converge_state_and_update'", "/home/oracle/logstash-6.1.1/logstash-

core/lib/logstash/agent.rb:90:in execute'", "/home/oracle/logstash-6.1.1/logstash-core/lib/logstash/runner.rb:343:inblock in execute'", "/home/oracle/logstash-6.1.1/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in `block in

initialize'"]}
[2018-01-18T08:21:10,210][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}

My Apologies for this huge error log, could you please help me overcoming these errors ?

Kind Regards
Bheeshma.

[Bheeshma]

Dear Experts,

I have given several attempts to make the logstash reading the file resides on the same server but ending up with several errors which I have no clue at all what they are meant to be

Here is the logstash.conf file contents from the location:
input {
file {
path => "/home/oracle/Downloads/App.LOG"
type => "file"
}
}

output {
elasticsearch
{
hosts => ["localhost:9600"]
index => "metrics-prod-%{+YYYY.MM}"
}
stdout { codec => rubydebug }
}

Here is the logstash.yml file contents from the location:

path.data: /home/oracle/Downloads/
path.config: /etc/logstash/conf.d/*.conf
path.logs: /home/oracle/logstash-6.1.1/logs

After running the following command - ./logstash -f /etc/logstash/conf.d/logstash.conf

its just a snippet of the error log:

[2018-02-06T10:27:52,703][ERROR][logstash.agent ] An exception happened when converging configuration {:exception=>LogStash::Error, :message=>"Don't know how to handle Java::JavaLang::NoClassDefFoundError for LogStash::PipelineAction::Create/pipeline_id:main", :backtrace=>["/home/oracle/logstash-6.1.1/logstash-core/lib/logstash/converge_result.rb:27:in create'", "/home/oracle/logstash-6.1.1/logstash-core/lib/logstash/converge_result.rb:67:inadd'", "/home/oracle/logstash-6.1.1/logstash-core/lib/logstash/agent.rb:347:in block in converge_state'", "/home/oracle/logstash-6.1.1/logstash-core/lib/logstash/agent.rb:141:inwith_pipelines'", "/home/oracle/logstash-6.1.1/logstash-core/lib/logstash/agent.rb:332:in block in converge_state'", "org/jruby/RubyArray.java:1734:ineach'", "/home/oracle/logstash-6.1.1/logstash-core/lib/logstash/agent.rb:319:in converge_state'", "/home/oracle/logstash-6.1.1/logstash-core/lib/logstash/agent.rb:166:inblock in converge_state_and_update'", "/home/oracle/logstash-6.1.1/logstash-core/lib/logstash/agent.rb:141:in with_pipelines'", "/home/oracle/logstash-6.1.1/logstash-core/lib/logstash/agent.rb:164:inconverge_state_and_update'", "/home/oracle/logstash-6.1.1/logstash-core/lib/logstash/agent.rb:90:in execute'", "/home/oracle/logstash-6.1.1/logstash-core/lib/logstash/runner.rb:343:inblock in execute'", "/home/oracle/logstash-6.1.1/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in `block in initialize'"]}

Please, kindly advise.

Regards
Bheeshma.

[Bheeshma]

When I ran the following command, I got the below message..seems Logstash configuration issues but now sure how to over come this, please advise.

[oracle@server logstash-6.1.1]$ bin/logstash --path.settings /etc/logstash -f /etc/logstash/conf.d/logstash.conf --config.test_and_exit
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /etc/logstash/log4j2.properties. Using default config which logs errors to the console
Unhandled Java exception: java.lang.NoClassDefFoundError
java.lang.NoClassDefFoundError: null