Input json file into elasticsearch via logstash

Elastic Stack Logstash
1

HI,
I am trying to input json file into elasticsearch but I get this error:

Could not find log4j2 configuration at path /logstash-5.4.0/config/log4j2.properties. Using default config which logs to console
12:27:10.702 [main] FATAL logstash.runner - An unexpected error occurred! {:error=>#<ArgumentError: Setting "xpack.monitoring.elasticsearch.username" hasn't been registered>, :backtrace=>["C:/logstash-5.4.0/logstash-core/lib/logstash/settings.rb:29:in get_setting'", "C:/logstash-5.4.0/logstash-core/lib/logstash/settings.rb:61:inset_value'", "C:/logstash-5.4.0/logstash-core/lib/logstash/settings.rb:80:in merge'", "org/jruby/RubyHash.java:1342:ineach'", "C:/logstash-5.4.0/logstash-core/lib/logstash/settings.rb:80:in merge'", "C:/logstash-5.4.0/logstash-core/lib/logstash/settings.rb:129:invalidate_all'", "C:/logstash-5.4.0/logstash-core/lib/logstash/runner.rb:217:in execute'", "C:/logstash-5.4.0/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:67:inrun'", "C:/logstash-5.4.0/logstash-core/lib/logstash/runner.rb:185:in run'", "C:/logstash-5.4.0/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:132:inrun'", "C:\logstash-5.4.0\lib\bootstrap\environment.rb:71:in `(root)'"]}

I have removed x-pack plugin.

my logstash.conf file is:
input
{
file
{
path => ["file:///C:/Users/EEBS/Desktop/json/shakespeare.json"]
start_position => "beginning"
sincedb_path => "/dev/null"
exclude => "*.gz"
}
}

filter
{
mutate
{
replace => [ "message", "%{message}" ]
gsub => [ 'message','\n','']
}
if [message] =~ /^{.*}$/
{
json { source => message }
}

}

output
{
elasticsearch {
protocol => "http"
codec => json
host => "localhost"
index => "json"
embedded => true
}

stdout { codec => rubydebug }

}

2

What does /logstash-5.4.0/config/logstash.yml look like? Please always post configuration files as preformatted text.

The second line of the log has been damaged when you posted. What actually comes after ":error=>#"? You can just repost the log as preformatted text.

3 
indent preformatted text by 4 spaces

@magnusbaeck
My .conf file:

input
{
file
{
path => ["file:///C:/Users/EEBS/Desktop/json/shakespeare.json"]
start_position => "beginning"
sincedb_path => "/dev/null"
exclude => "*.gz"
}
}

filter
{
mutate
{
replace => [ "message", "%{message}" ]
gsub => [ 'message','\n','']
}
if [message] =~ /^{.*}$/
{
json { source => message }
}

}

output
{
elasticsearch {
protocol => "http"
codec => json
host => "localhost"
index => "json"
embedded => true
}

stdout { codec => rubydebug }
}

and error log is:

12:27:10.702 [main] FATAL logstash.runner - An unexpected error occurred! {:error=>#, :backtrace=>["C:/logstash-5.4.0/logstash-core/lib/logstash/settings.rb:29:in get_setting'", "C:/logstash-5.4.0/logstash-core/lib/logstash/settings.rb:61:inset_value'", "C:/logstash-5.4.0/logstash-core/lib/logstash/settings.rb:80:in merge'", "org/jruby/RubyHash.java:1342:ineach'", "C:/logstash-5.4.0/logstash-core/lib/logstash/settings.rb:80:in merge'", "C:/logstash-5.4.0/logstash-core/lib/logstash/settings.rb:129:invalidate_all'", "C:/logstash-5.4.0/logstash-core/lib/logstash/runner.rb:217:in execute'", "C:/logstash-5.4.0/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:67:inrun'", "C:/logstash-5.4.0/logstash-core/lib/logstash/runner.rb:185:in run'", "C:/logstash-5.4.0/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:132:inrun'", "C:\logstash-5.4.0\lib\bootstrap\environment.rb:71:in `(root)'"]}

4

Still damaged. What comes after ":error=>#"?!

5

I am getting this error log :
indent preformatted text by 4 spaces
Could not find log4j2 configuration at path /logstash-5.4.0/config/log4j2.properties. Using default config which logs to console
17:15:41.137 [LogStash::Runner] ERROR logstash.outputs.elasticsearch - Unknown setting 'protocol' for elasticsearch
17:15:41.140 [LogStash::Runner] ERROR logstash.outputs.elasticsearch - Unknown setting 'host' for elasticsearch
17:15:41.141 [LogStash::Runner] ERROR logstash.outputs.elasticsearch - Unknown setting 'embedded' for elasticsearch
17:15:41.149 [LogStash::Runner] ERROR logstash.agent - Cannot create pipeline {:reason=>"Something is wrong with your configuration."}

6

The three mentioned settings for the elasticsearch output were deprecated as of Logstash 2.0. See the documentation for your version of Logstash for a list of available settings.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.