Java Logs patterns

Elastic Stack Logstash
1

please advice - is this format correct to review structured java log?

input {
beats {
port =>5044
}
}

output {
elasticsearch {
hosts => "localhost:9200"
manage_template => false
index => ["NUMBER:thread", "message", "%{TIMESTAMP_ISO8601:logdate} %{LOGLEVEL:loglevel} (?.*)"]
document_type => "%{[@metadata][type]}"
}
}

2

No. You've somehow conflated the grok filter and the elasticsearch output. For now I suggest you leave the index option alone entirely and that you add a grok filter. Your match expression doesn't look completely unreasonable (depending on what the log looks like; there's no standard format for Java application logs).

3

Thanks
please assist with this one , please bare with me still a newbie in ELK
I am trying to get - pattern : Thread>>Message>>LogLevel

input{
beats{
port=>5044
type=>"*.log"
codec=>
multiline{
charset=>"ISO-8859-1"
pattern=>"^%{TIMESTAMP_ISO8601}"
max_lines=>1000
negate=>true
what=>"previous"
}
}
}filter{
mutate{
gsub=>["message","\r",""]
}grok{
patterns_dir=>"./patterns"
match => { message => ["%{TIMESTAMP_ISO8601:timestamp}","%{LOGLEVEL1:loglevel}","%{SKYEXCEPTION_TYPE:exception}"]}
}date{
match=>["timestamp","yyyy-MM-dd HH:mm:ss,SSS"]
remove_field=>["timestamp"]
}
}output{
elasticsearch{
hosts=>"localhos:9200"
}
}

4

I am trying to get - pattern : Thread>>Message>>LogLevel

I don't know what you mean by this. It would be easier to help if we knew what your messages look like.

General comments:

  • Perform multiline processing on the Beats side. Don't do it in Logstash.
  • The hostname in the elasticsearch hosts option is misspellt.
  • You're listing three different grok expressions in your grok filter. Don't do that; write a single grok expression that captures everything you want.
  • To avoid surprises use an absolute path to the pattern file in the grok filter.
5

I am trying to get - pattern : Thread>>Message>>LogLevel
i want my kibana to display / visualize my logs in this fashion - Thread.message.loglevel.date

6

can you send me default configs and beats configs

  1. logstash.conf
  2. logstash.yml
7

I am trying to get - pattern : Thread>>Message>>LogLevel

As I said, I don't understand what you mean by this. Instead of describing what you want to do please use concrete examples. What does a log message look like? Which fields do you want in Kibana?

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.