Grok for JVM console log

I want to parse the JVM console logs ignoring the initial load and just considering the actual logs which starts with [0;31m2016-09-29 04:00:03,264 timestamp.

I am not able to understand the starting point and how to ignore the initial JVM loading.

I am not able to put the logs as it is beyond the recommended size.

Please help

--> Wrapper Started as Daemon
Java Service Wrapper Standard Edition 64-bit 3.5.24
  Copyright (C) 1999-2014 Tanuki Software, Ltd. All Rights Reserved.
    http://wrapper.tanukisoftware.com
  Licensed to xxxxx software - www.xxxxx.com for xxxxx Platform

Launching a JVM...
Install Directory resolved to[/apps/xxxxxxxxxx/appserveragent/ver4.2.1.8]
TERM trapped.  Shutting down.
Full Agent Registration Info Resolver using selfService [false]
Full Agent Registration Info Resolver using selfService [false]
Full Agent Registration Info Resolver using application name [prlus-perf]
Full Agent Registration Info Resolver using tier name [Storefront-AppServers]
Full Agent Registration Info Resolver using node name [xxxxxx.xxxxxx.com]
[Thread-0] Thu Sep 29 00:46:06 EDT 2016[DEBUG]: AgentInstallManager - Full Agent Registration Info Resolver is running
[Thread-0] Thu Sep 29 00:46:06 EDT 2016[INFO]: AgentInstallManager - Full Agent Registration Info Resolver using selfService [false]
[Thread-0] Thu Sep 29 00:46:06 EDT 2016[INFO]: AgentInstallManager - Full Agent Registration Info Resolver using selfService [false]
[Thread-0] Thu Sep 29 00:46:06 EDT 2016[INFO]: AgentInstallManager - Full Agent Registration Info Resolver using application name [prlus-perf]
[Thread-0] Thu Sep 29 00:46:06 EDT 2016[INFO]: AgentInstallManager - Full Agent Registration Info Resolver using tier name [Storefront-AppServers]
[Thread-0] Thu Sep 29 00:46:06 EDT 2016[INFO]: AgentInstallManager - Full Agent Registration Info Resolver using node name [xxxxxxxxxx]
[Thread-0] Thu Sep 29 00:46:06 EDT 2016[DEBUG]: AgentInstallManager - Full Agent Registration Info Resolver finished running
[Thread-0] Thu Sep 29 00:46:06 EDT 2016[INFO]: AgentInstallManager - Agent runtime directory set to [/apps/xxxxxxxxxx/appserveragent/ver4.2.1.8]
[Thread-0] Thu Sep 29 00:46:06 EDT 2016[INFO]: AgentInstallManager - Agent node directory set to [xxxxxxxxxx]
[Thread-0] Thu Sep 29 00:46:07 EDT 2016[INFO]: JavaAgent - Using Java Agent Version [Server Agent v4.2.1.8 GA #11326 rb3b8138e547e0d2dd1cee3f6b3ef9ab6e14fdf33 113-4.2.1.next-build]
[Thread-0] Thu Sep 29 00:46:07 EDT 2016[INFO]: JavaAgent - Running IBM Java Agent [No]
[Thread-0] Thu Sep 29 00:46:07 EDT 2016[INFO]: JavaAgent - Java Agent Directory [/apps/xxxxxxxxxx/appserveragent/ver4.2.1.8]
[Thread-0] Thu Sep 29 00:46:07 EDT 2016[INFO]: JavaAgent - Java Agent AppAgent directory [/apps/xxxxxxxxxx/appserveragent/ver4.2.1.8]
Agent Logging Directory [/apps/xxxxxxxxxx/appserveragent/ver4.2.1.8/logs/xxxxxxxxxx]

Launching a JVM...
Install Directory resolved to[/apps/xxxxxxxxxx/appserveragent/ver4.2.1.8]
Full Agent Registration Info Resolver using selfService [false]
Full Agent Registration Info Resolver using selfService [false]
Full Agent Registration Info Resolver using application name [prlus-perf]
Full Agent Registration Info Resolver using tier name [Storefront-AppServers]
Full Agent Registration Info Resolver using node name [hybsfperfasgbc66568d.abcdefg.com]
[Thread-0] Thu Sep 29 00:51:24 EDT 2016[DEBUG]: AgentInstallManager - Full Agent Registration Info Resolver is running
[Thread-0] Thu Sep 29 00:51:24 EDT 2016[INFO]: AgentInstallManager - Full Agent Registration Info Resolver using selfService [false]
[Thread-0] Thu Sep 29 00:51:24 EDT 2016[INFO]: AgentInstallManager - Full Agent Registration Info Resolver using selfService [false]
[Thread-0] Thu Sep 29 00:51:24 EDT 2016[INFO]: AgentInstallManager - Full Agent Registration Info Resolver using application name [prlus-perf]
[Thread-0] Thu Sep 29 00:51:24 EDT 2016[INFO]: AgentInstallManager - Full Agent Registration Info Resolver using tier name [Storefront-AppServers]
[Thread-0] Thu Sep 29 00:51:24 EDT 2016[INFO]: AgentInstallManager - Full Agent Registration Info Resolver using node name [hybsfperfasgbc66568d.abcdefg.com]
[Thread-0] Thu Sep 29 00:51:24 EDT 2016[DEBUG]: AgentInstallManager - Full Agent Registration Info Resolver finished running
[Thread-0] Thu Sep 29 00:51:24 EDT 2016[INFO]: AgentInstallMa

e[0;31m2016-09-29 04:00:03,264 ERROR [Thread: xxxxxHTTP91] [Method: getContentPageForRequest] [xx.xx.xx.xxx] [com.abcdefg.storefront.controllers.pages.RLDefaultPageController] ERR_DEFAULTPAGE_002 | CMSItem for the id not Found.
 e[mde.xxxxx.platform.cms2.exceptions.CMSItemNotFoundException: No page with label [/abcdefg/_ui/appversion.html] found.
	at de.xxxxx.platform.cms2.servicelayer.services.impl.DefaultCMSPageService.getPageForLabel(DefaultCMSPageService.java:561)
	at com.abcdefg.storefront.controllers.pages.RLDefaultPageController.getContentPageForRequest(RLDefaultPageController.java:262)
	at com.abcdefg.storefront.controllers.pages.RLDefaultPageController.get(RLDefaultPageController.java:123)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:221)
	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:137)
	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:110)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:776)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:705)
	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)`

I have provided some snapshot of the log

Use a multiline codec to express "unless the line begins with the thread name and a timestamp, join the current line with the previous". As for the grok expression to extract timestamp, thread name, log level etc, where are you stuck?

I am not able figure out where to start from because initial part of the log is all the JVM initializing logs which i dont want. I want to log from

[0;31m2016-09-29 04:00:03,264 ERROR [Thread: xxxxxHTTP91] [Method: getContentPageForRequest] [xx.xx.xx.xxx] [com.abcdefg.storefront.controllers.pages.RLDefaultPageController] ERR_DEFAULTPAGE_002 | CMSItem for the id not Found.
e[mde.xxxxx.platform.cms2.exceptions.CMS

Ignore the beginning of the log for now and focus on getting the multiline and grok right.

how do i segregate the below

0;31m2016-09-29 05:27:33,992

Isn't "0;31m" a fixed string? In that case I'm not sure what the problem is. You should be able to use

(0;31m)?%{TIMESTAMP_ISO8601:timestamp}

to match this (with "0;31m" being optional) and extract the timestamp into a timestamp field.