Not able to post data


(Prateek Kshtriya) #1

Hi All,

I am not able to post data using logstash 5.0.2 to elasticsearch5.0.1. I am getting below error-

\Logstash\logstash-5.0.2\bin>logstash -f apache.conf
Could not find log4j2 configuration at path /ELK/Logstash/logstash-5.0.2/config/
log4j2.properties. Using default config which logs to console
16:51:19.125 [[main]-pipeline-manager] INFO logstash.outputs.elasticsearch - El
asticsearch pool URLs updated {:changes=>{:removed=>[], :added=>["http://127.0.0
.1:9200"]}}
16:51:19.125 [[main]-pipeline-manager] INFO logstash.outputs.elasticsearch - Us
ing mapping template from {:path=>nil}
16:51:19.235 [[main]-pipeline-manager] INFO logstash.outputs.elasticsearch - At
tempting to install template {:manage_template=>{"template"=>"logstash-", "vers
ion"=>50001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"defaul
t
"=>{"_all"=>{"enabled"=>true, "norms"=>false}, "dynamic_templates"=>[{"message
_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"
type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"
", "match_mappin
g_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keywo
rd"=>{"type"=>"keyword"}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date", "in
clude_in_all"=>false}, "@version"=>{"type"=>"keyword", "include_in_all"=>false},
"geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"t
ype"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"h
alf_float"}}}}}}}}
16:51:19.245 [[main]-pipeline-manager] INFO logstash.outputs.elasticsearch - Ne
w Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["12
7.0.0.1"]}
16:51:19.330 [[main]-pipeline-manager] ERROR logstash.agent - Pipeline aborted d
ue to error {:exception=>#<Grok::PatternError: pattern %{HTTPNEWDATE:time} not d
efined>, :backtrace=>["D:/ELK/Logstash/logstash-5.0.2/vendor/bundle/jruby/1.9/ge
ms/jls-grok-0.11.4/lib/grok-pure.rb:123:in compile'", "org/jruby/RubyKernel.jav a:1479:inloop'", "D:/ELK/Logstash/logstash-5.0.2/vendor/bundle/jruby/1.9/gems/
jls-grok-0.11.4/lib/grok-pure.rb:93:in compile'", "D:/ELK/Logstash/logstash-5.0 .2/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-3.3.0/lib/logstash/filters/ grok.rb:272:inregister'", "org/jruby/RubyArray.java:1613:in each'", "D:/ELK/L ogstash/logstash-5.0.2/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-3.3.0/l ib/logstash/filters/grok.rb:267:inregister'", "org/jruby/RubyHash.java:1342:in
each'", "D:/ELK/Logstash/logstash-5.0.2/vendor/bundle/jruby/1.9/gems/logstash- filter-grok-3.3.0/lib/logstash/filters/grok.rb:262:inregister'", "D:/ELK/Logst
ash/logstash-5.0.2/logstash-core/lib/logstash/pipeline.rb:197:in start_workers' ", "org/jruby/RubyArray.java:1613:ineach'", "D:/ELK/Logstash/logstash-5.0.2/lo
gstash-core/lib/logstash/pipeline.rb:197:in start_workers'", "D:/ELK/Logstash/l ogstash-5.0.2/logstash-core/lib/logstash/pipeline.rb:153:inrun'", "D:/ELK/Logs
tash/logstash-5.0.2/logstash-core/lib/logstash/agent.rb:250:in `start_pipeline'"
]}
16:51:19.393 [Api Webserver] INFO logstash.agent - Successfully started Logstas
h API endpoint {:port=>9600}
16:51:22.349 [LogStash::Runner] WARN logstash.agent - stopping pipeline {:id=>"
main"}

PFB the config file snippet-

input {
file {
path => "tomcat7\logs\dashboard.log"
start_position => "beginning"
}
}

filter {

multiline {
pattern => "%{YEAR}-%{MONTH}"
what => "next"
}
grok {
match => { "message" => " %{WORD:SEVERITYLEVEL} " }
}
grok {

       match => { "message" => "  %{GREEDYDATA:ServerMessage}" }
      }
grok {
		match => { "message" => "] %{GREEDYDATA:ClassName}" }
        }
grok {
               match => { "message" => "\[%{PROG:ServiceName}" }
      }

}

output {
if "ERROR" in [SEVERITYLEVEL]{
elasticsearch {
action => "index"
codec => "json"
index => "dashboardlogs"
}
}
}

Please suggest if I am missing something as I had used logstash2.1.0 earlier.

Regards,
Prateek Divya


(Magnus B├Ąck) #2

Here's the error message:

Grok::PatternError: pattern %{HTTPNEWDATE:time} not defined

You have a grok filter somewhere that attempt to use the non-existing HTTPNEWDATE pattern.


(Prateek Kshtriya) #3

Thank you. It worked on adding appropriate grok pattern.

Regards,
Prateek Divya


(system) #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.