Keep newline formating with multiline

Elastic Stack Logstash
1

Here is my config file. I am processing java logs. The errors do come into Kibana but are all one long chunk of text. Is there a way to have the entry come in as one event but keep the formating so it is more readable?

input {
file {
path => [
"/home/glassfish/gf-logs/logs/server.log"
]
codec =>
multiline {
pattern => "^[\d{4}"
negate => "true"
what => "previous"
}
start_position => "beginning"
}
}

filter {
mutate {
remove_field => [ "@version" ]
remove_field => [ "path" ]
remove_field => [ "_score" ]
gsub => ["message", "\n", ""]
}
grok {
match => {
"message" => "[%{DATA:server_version}] [%{LOGLEVEL}] [] [%{JAVACLASS}] [%{DATA:thread}] [%{DATA}] [%{DATA}] [[%{DATA:logmessage} ]]"
}
}
date {
match => [
"timestamp", "ISO8601"
]
target => "@timestamp"
}
kv {
source => "logmessage"
}
}

output {
elasticsearch {
hosts => [ "192.168.15.242:9200" ]
index => "beta-%{+YYYY.MM.dd}"
}
}

2

I don't understand. What do your events currently look like? What would you like them to look like instead?

(Side note: For efficiency reasons you should never use more DATA or GREEDYDATA patterns than necessary. You don't need more than one in this case.)

3

Thanks for your help
----------HOW ERROR SHOWS UP ON SERVER-----------

[2017-10-19T15:21:06.871-0700] [Payara 4.1] [WARNING] [] [org.apache.jasper.runtime.TldScanner] [tid: _ThreadID=17 _ThreadName=RunLevelControllerThread-1508451653051] [timeMillis: 1508451
666871] [levelValue: 900] [[
PWC6351: In TLD scanning, the supplied resource file:/home/glassfish/payara41/glassfish/nodes/js-beta03/js-beta03-catseye/applications/Catseye-Beta2/lib/xercesImpl.jar does not exist
java.io.FileNotFoundException: /home/glassfish/payara41/glassfish/nodes/js-beta03/js-beta03-catseye/applications/Catseye-Beta2/lib/xercesImpl.jar (No such file or directory)
at java.util.zip.ZipFile.open(Native Method)
at java.util.zip.ZipFile.(ZipFile.java:219)
at java.util.zip.ZipFile.(ZipFile.java:149)
at java.util.jar.JarFile.(JarFile.java:166)
at java.util.jar.JarFile.(JarFile.java:103)
at sun.net.www.protocol.jar.URLJarFile.(URLJarFile.java:93)
at sun.net.www.protocol.jar.URLJarFile.getJarFile(URLJarFile.java:69)
at sun.net.www.protocol.jar.JarFileFactory.get(JarFileFactory.java:99)
at sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java:122)
at sun.net.www.protocol.jar.JarURLConnection.getJarFile(JarURLConnection.java:89)

----------HOW ERROR SHOWS UP IN KIBANA-----------

[2017-10-19T15:21:06.871-0700] [Payara 4.1] [WARNING] [] [org.apache.jasper.runtime.TldScanner] [tid: _ThreadID=17 _ThreadName=RunLevelControllerThread-1508451653051] [timeMillis: 1508451666871] [levelValue: 900] [[  PWC6351: In TLD scanning, the supplied resource file:/home/glassfish/payara41/glassfish/nodes/js-beta03/js-beta03-catseye/applications/Catseye-Beta2/lib/xercesImpl.jar does not existjava.io.FileNotFoundException: /home/glassfish/payara41/glassfish/nodes/js-beta03/js-beta03-catseye/applications/Catseye-Beta2/lib/xercesImpl.jar (No such file or directory)	at java.util.zip.ZipFile.open(Native Method)	at java.util.zip.ZipFile.<init>(ZipFile.java:219)	at java.util.zip.ZipFile.<init>(ZipFile.java:149)	at java.util.jar.JarFile.<init>(JarFile.java:166)	at java.util.jar.JarFile.<init>(JarFile.java:103)	at sun.net.www.protocol.jar.URLJarFile.<init>(URLJarFile.java:93)	at sun.net.www.protocol.jar.URLJarFile.getJarFile(URLJarFile.java:69)	at sun.net.www.protocol.jar.JarFileFactory.get(JarFileFactory.java:99)	at sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java:122)	at sun.net.www.protocol.jar.JarURLConnection.getJarFile(JarURLConnection.java:89)	at org.apache.jasper.runtime.TldScanner.scanJar(TldScanner.java:445)	at org.apache.jasper.runtime.TldScanner.scanJars(TldScanner.java:697)	at org.apache.jasper.runtime.TldScanner.scanTlds(TldScanner.java:353)	at org.apache.jasper.runtime.TldScanner.onStartup(TldScanner.java:242)	at org.apache.catalina.core.StandardContext.callServletContainerInitialize
4

I figured it out, this line was removing the newline and messing up the formatting.
gsub => ["message", "\n", ""]

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.