Not able to post data

Hi All,

I am not able to post data using logstash 5.0.2 to elasticsearch5.0.1. I am getting below error-

\Logstash\logstash-5.0.2\bin>logstash -f apache.conf
Could not find log4j2 configuration at path /ELK/Logstash/logstash-5.0.2/config/
log4j2.properties. Using default config which logs to console
16:51:19.125 [[main]-pipeline-manager] INFO logstash.outputs.elasticsearch - El
asticsearch pool URLs updated {:changes=>{:removed=>[], :added=>["http://127.0.0
.1:9200"]}}
16:51:19.125 [[main]-pipeline-manager] INFO logstash.outputs.elasticsearch - Us
ing mapping template from {:path=>nil}
16:51:19.235 [[main]-pipeline-manager] INFO logstash.outputs.elasticsearch - At
tempting to install template {:manage_template=>{"template"=>"logstash-", "vers
ion"=>50001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"defaul
t
"=>{"_all"=>{"enabled"=>true, "norms"=>false}, "dynamic_templates"=>[{"message
_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"
type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"
", "match_mappin
g_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keywo
rd"=>{"type"=>"keyword"}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date", "in
clude_in_all"=>false}, "@version"=>{"type"=>"keyword", "include_in_all"=>false},
"geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"t
ype"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"h
alf_float"}}}}}}}}
16:51:19.245 [[main]-pipeline-manager] INFO logstash.outputs.elasticsearch - Ne
w Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["12
7.0.0.1"]}
16:51:19.330 [[main]-pipeline-manager] ERROR logstash.agent - Pipeline aborted d
ue to error {:exception=>#<Grok::PatternError: pattern %{HTTPNEWDATE:time} not d
efined>, :backtrace=>["D:/ELK/Logstash/logstash-5.0.2/vendor/bundle/jruby/1.9/ge
ms/jls-grok-0.11.4/lib/grok-pure.rb:123:in compile'", "org/jruby/RubyKernel.jav a:1479:inloop'", "D:/ELK/Logstash/logstash-5.0.2/vendor/bundle/jruby/1.9/gems/
jls-grok-0.11.4/lib/grok-pure.rb:93:in compile'", "D:/ELK/Logstash/logstash-5.0 .2/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-3.3.0/lib/logstash/filters/ grok.rb:272:inregister'", "org/jruby/RubyArray.java:1613:in each'", "D:/ELK/L ogstash/logstash-5.0.2/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-3.3.0/l ib/logstash/filters/grok.rb:267:inregister'", "org/jruby/RubyHash.java:1342:in
each'", "D:/ELK/Logstash/logstash-5.0.2/vendor/bundle/jruby/1.9/gems/logstash- filter-grok-3.3.0/lib/logstash/filters/grok.rb:262:inregister'", "D:/ELK/Logst
ash/logstash-5.0.2/logstash-core/lib/logstash/pipeline.rb:197:in start_workers' ", "org/jruby/RubyArray.java:1613:ineach'", "D:/ELK/Logstash/logstash-5.0.2/lo
gstash-core/lib/logstash/pipeline.rb:197:in start_workers'", "D:/ELK/Logstash/l ogstash-5.0.2/logstash-core/lib/logstash/pipeline.rb:153:inrun'", "D:/ELK/Logs
tash/logstash-5.0.2/logstash-core/lib/logstash/agent.rb:250:in `start_pipeline'"
]}
16:51:19.393 [Api Webserver] INFO logstash.agent - Successfully started Logstas
h API endpoint {:port=>9600}
16:51:22.349 [LogStash::Runner] WARN logstash.agent - stopping pipeline {:id=>"
main"}

PFB the config file snippet-

input {
file {
path => "tomcat7\logs\dashboard.log"
start_position => "beginning"
}
}

filter {

multiline {
pattern => "%{YEAR}-%{MONTH}"
what => "next"
}
grok {
match => { "message" => " %{WORD:SEVERITYLEVEL} " }
}
grok {

       match => { "message" => "  %{GREEDYDATA:ServerMessage}" }
      }
grok {
		match => { "message" => "] %{GREEDYDATA:ClassName}" }
        }
grok {
               match => { "message" => "\[%{PROG:ServiceName}" }
      }

}

output {
if "ERROR" in [SEVERITYLEVEL]{
elasticsearch {
action => "index"
codec => "json"
index => "dashboardlogs"
}
}
}

Please suggest if I am missing something as I had used logstash2.1.0 earlier.

Regards,
Prateek Divya

Here's the error message:

Grok::PatternError: pattern %{HTTPNEWDATE:time} not defined

You have a grok filter somewhere that attempt to use the non-existing HTTPNEWDATE pattern.

Thank you. It worked on adding appropriate grok pattern.

Regards,
Prateek Divya

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.