Hi All,
I am not able to post data using logstash 5.0.2 to elasticsearch5.0.1. I am getting below error-
\Logstash\logstash-5.0.2\bin>logstash -f apache.conf
Could not find log4j2 configuration at path /ELK/Logstash/logstash-5.0.2/config/
log4j2.properties. Using default config which logs to console
16:51:19.125 [[main]-pipeline-manager] INFO logstash.outputs.elasticsearch - El
asticsearch pool URLs updated {:changes=>{:removed=>[], :added=>["http://127.0.0
.1:9200"]}}
16:51:19.125 [[main]-pipeline-manager] INFO logstash.outputs.elasticsearch - Us
ing mapping template from {:path=>nil}
16:51:19.235 [[main]-pipeline-manager] INFO logstash.outputs.elasticsearch - At
tempting to install template {:manage_template=>{"template"=>"logstash-", "vers
ion"=>50001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"defaul
t"=>{"_all"=>{"enabled"=>true, "norms"=>false}, "dynamic_templates"=>[{"message
_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"
type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"", "match_mappin
g_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keywo
rd"=>{"type"=>"keyword"}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date", "in
clude_in_all"=>false}, "@version"=>{"type"=>"keyword", "include_in_all"=>false},
"geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"t
ype"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"h
alf_float"}}}}}}}}
16:51:19.245 [[main]-pipeline-manager] INFO logstash.outputs.elasticsearch - Ne
w Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["12
7.0.0.1"]}
16:51:19.330 [[main]-pipeline-manager] ERROR logstash.agent - Pipeline aborted d
ue to error {:exception=>#<Grok::PatternError: pattern %{HTTPNEWDATE:time} not d
efined>, :backtrace=>["D:/ELK/Logstash/logstash-5.0.2/vendor/bundle/jruby/1.9/ge
ms/jls-grok-0.11.4/lib/grok-pure.rb:123:in compile'", "org/jruby/RubyKernel.jav a:1479:in
loop'", "D:/ELK/Logstash/logstash-5.0.2/vendor/bundle/jruby/1.9/gems/
jls-grok-0.11.4/lib/grok-pure.rb:93:in compile'", "D:/ELK/Logstash/logstash-5.0 .2/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-3.3.0/lib/logstash/filters/ grok.rb:272:in
register'", "org/jruby/RubyArray.java:1613:in each'", "D:/ELK/L ogstash/logstash-5.0.2/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-3.3.0/l ib/logstash/filters/grok.rb:267:in
register'", "org/jruby/RubyHash.java:1342:in
each'", "D:/ELK/Logstash/logstash-5.0.2/vendor/bundle/jruby/1.9/gems/logstash- filter-grok-3.3.0/lib/logstash/filters/grok.rb:262:in
register'", "D:/ELK/Logst
ash/logstash-5.0.2/logstash-core/lib/logstash/pipeline.rb:197:in start_workers' ", "org/jruby/RubyArray.java:1613:in
each'", "D:/ELK/Logstash/logstash-5.0.2/lo
gstash-core/lib/logstash/pipeline.rb:197:in start_workers'", "D:/ELK/Logstash/l ogstash-5.0.2/logstash-core/lib/logstash/pipeline.rb:153:in
run'", "D:/ELK/Logs
tash/logstash-5.0.2/logstash-core/lib/logstash/agent.rb:250:in `start_pipeline'"
]}
16:51:19.393 [Api Webserver] INFO logstash.agent - Successfully started Logstas
h API endpoint {:port=>9600}
16:51:22.349 [LogStash::Runner] WARN logstash.agent - stopping pipeline {:id=>"
main"}
PFB the config file snippet-
input {
file {
path => "tomcat7\logs\dashboard.log"
start_position => "beginning"
}
}
filter {
multiline {
pattern => "%{YEAR}-%{MONTH}"
what => "next"
}
grok {
match => { "message" => " %{WORD:SEVERITYLEVEL} " }
}
grok {
match => { "message" => " %{GREEDYDATA:ServerMessage}" }
}
grok {
match => { "message" => "] %{GREEDYDATA:ClassName}" }
}
grok {
match => { "message" => "\[%{PROG:ServiceName}" }
}
}
output {
if "ERROR" in [SEVERITYLEVEL]{
elasticsearch {
action => "index"
codec => "json"
index => "dashboardlogs"
}
}
}
Please suggest if I am missing something as I had used logstash2.1.0 earlier.
Regards,
Prateek Divya