Can't fix _grokparsefailure

Elastic Stack Logstash
1

Hi

I can't seem to get my grok filter to work.

I want to use filebeat to parse a log file, send it to Logstash, parse it and send it to Elasticsearch.

My filebeat.yml:
filebeat:
prospectors:
- type: log
paths:
- /opt/tomcat/logs/intr4.log
multiline:
pattern: '^\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}'
match: after
negate: true

output.logstash:
hosts: ["slog01v:5044"]

My logstash conf file in conf.d
input {
beats {
port => 5044
}
}
filter {
grok {
match => ["message", "%{TIMESTAMP_ISO8601:timestamp} [%{JAVAFILE:thread}] %{LOGLEVEL:loglevel}%{SPACE}%{JAVACLASS:classname} - %{DATA:logmessage} "]
}
date {
match => ["timestamp", "YYYY-MM-dd HH:mm:ss"]
remove_field => ["timestamp"]
}
}

output {
elasticsearch {
hosts => "shad02:9200"
index => "smartgrid-%{+YYYY.MM.dd}"
}
stdout { codec => rubydebug }
}

The log file contains:
2018-12-06 13:41:28 [http-nio-8080-exec-67] INFO o.a.c.spring.CamelContextFactoryBean - Bridging Camel and Spring property placeholder configurer with id: bridgePropertyPlaceholder
2018-12-06 13:41:28 [ActiveMQ Task-1] INFO o.a.a.t.failover.FailoverTransport - Successfully connected to tcp://DINT01v.elnet.local:61616
2018-12-06 13:41:29 [http-nio-8080-exec-67] INFO o.a.c.i.c.DefaultTypeConverter - Loaded 196 type converters
2018-12-06 13:41:29 [http-nio-8080-exec-67] INFO o.a.camel.spring.SpringCamelContext - Apache Camel 2.19.3 (CamelContext: intr-eip-e66-producer-bpm-rsm012-session) is starting
2018-12-06 13:41:29 [http-nio-8080-exec-67] INFO o.a.c.m.ManagedManagementStrategy - JMX is enabled
2018-12-06 13:41:29 [http-nio-8080-exec-67] INFO o.a.c.i.DefaultRuntimeEndpointRegistry - Runtime endpoint registry is in extended mode gathering usage statistics of all incoming and outgoing endpoints (cache limit: 1000)

Can somebody please help me.

2

You need to escape the square brackets around the JAVAFILE:

\[%{JAVAFILE:thread}\]

Are you using a custom pattern? Because JAVAFILE and JAVACLASS do not appear to be standard.

3

I see that I lost something in the copy of the code, it is already there:

match => ["message", "%{TIMESTAMP_ISO8601:timestamp} \[%{JAVAFILE:thread}\] %{LOGLEVEL:loglevel}%{SPACE}%{JAVACLASS:classname} \- %{DATA:logmessage} "]
4

Where do you get "JAVAFILE" and "JAVACLASS" from?

5

Hi Lewis

I found those in som grok examples.

Should I use others?

6

Sorry you are correct, I have never seen those ones before!

This works for me using your logs:
%{TIMESTAMP_ISO8601:timestamp} \[%{JAVAFILE:thread}\] %{LOGLEVEL:loglevel} %{JAVACLASS:classname} - %{GREEDYDATA:logmessage}

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.