# New Install ELK 6.1 with Filebeat 6.1 for Tomcat Catalina Logs

**URL:** <https://discuss.elastic.co/t/new-install-elk-6-1-with-filebeat-6-1-for-tomcat-catalina-logs/113559>\
**Category:** Logstash\
**Created:** [December 29, 2017, 9:40am UTC](https://discuss.elastic.co/t/new-install-elk-6-1-with-filebeat-6-1-for-tomcat-catalina-logs/113559 "2017-12-29T09:40:07Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![vvsvinu](https://avatars.discourse-cdn.com/v4/letter/v/a88e57/32.png) [@vvsvinu](https://discuss.elastic.co/u/vvsvinu)\
**Post date:** [December 29, 2017, 9:40am UTC](https://discuss.elastic.co/t/new-install-elk-6-1-with-filebeat-6-1-for-tomcat-catalina-logs/113559/1 "2017-12-29T09:40:07Z")

</div>

Hai,

I am a newbie to ELK Stack. I have tried to configure the ELK Stack in a test machine in our LAB environment and was not able to succeed in parsing Apache Tomcat Logs. It would be great help if any one could assist me in completing it.

My purpose is to have a centralized log monitoring solution. Upon searching over Internet I have found about ELK Stack. I have installed the ELK Stack successfully and had also configured to ship "/var/log/messages" through my ELK Stack. But my actual requirement was to configure it to process Tomcat logs which I understand would require multi-line configuration which i had failed to configure. My requirements are

1. To have Tomcat logs filtered for valid TRACE, DEBUG, INFO, WARN, ERROR levels along with "/var/log/messages"
2. How to centralize theses tomcat logs running under different servers with different instances under ELK Stack.

Please help me as I had been trying to get it working for the past one week.

I have gone through the official documentation for Logstash and Filebeat. But it seems referring it is making me much more complicated. Hence it would kindly request to help me in configuration ELK Stack for tracking multiple server Tomcat Catalina logs. I have configured ELK Stack version 6.1 with File Beat 6.1 in a Linux Environment (RedHat 7.4). Please find my current working configurations for "/var/log/messages"..

############# Filebeat.yml #######################

filebeat.prospectors:

- type: log  
enabled: true  
paths:
  - /var/log/messages  
filebeat.config.modules:  
path: ${path.config}/modules.d/\*.yml  
reload.enabled: false  
setup.template.settings:  
index.number\_of\_shards: 3  
setup.kibana:  
host: "192.168.3.226:5601"  
output.logstash:  
hosts: ["192.168.3.226:5044"]

###################################################

############# Logstash.conf #######################

input {  
beats {  
host =\> "192.168.3.226"  
port =\> 5044  
}  
}  
filter {  
if [type] == "syslog" {  
grok {  
match =\> { "message" =\> "%{SYSLOGLINE}" }  
}  
date { match =\> ["timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"] }  
}  
}  
output {  
elasticsearch {  
hosts =\> ["192.168.3.226:9200"]  
index =\> "%{[@metadata][beat]}-%{+YYYY.MM.dd}"  
}  
stdout { codec =\> rubydebug }  
}

###################################################

Thank You

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [December 29, 2017, 9:59am UTC](https://discuss.elastic.co/t/new-install-elk-6-1-with-filebeat-6-1-for-tomcat-catalina-logs/113559/2 "2017-12-29T09:59:00Z")

</div>

FYI we’ve renamed ELK to the Elastic Stack, otherwise Beats and APM feel left out! 😉

Are you seeing anything via the `stdout` you have defined?

---

<div class="post-metadata">

**Author:** ![vvsvinu](https://avatars.discourse-cdn.com/v4/letter/v/a88e57/32.png) [@vvsvinu](https://discuss.elastic.co/u/vvsvinu)\
**Post date:** [December 30, 2017, 6:26am UTC](https://discuss.elastic.co/t/new-install-elk-6-1-with-filebeat-6-1-for-tomcat-catalina-logs/113559/3 "2017-12-30T06:26:17Z")

</div>

As per the current configuration, I am able to get "/var/log/messages" listed in the kibana interface, as in my current configuration the filebeat is transferring the logs to logstash from where it is getting filtered and the output is being displayed.

But when trying to traverse the catalina.out log, i have noticed that it contains logs listed in multiline format and hence needs to use multiline keyword in filebeat to get the Tomcat ERROR log and grok\_filter in logstash to get the logs listed in kibana.

But while attempting to do so I am not able to get the result. Moreover while trying to implement along with "/var/log/messages" also I get error.. Please help..😥😥

---

<div class="post-metadata">

**Author:** ![vvsvinu](https://avatars.discourse-cdn.com/v4/letter/v/a88e57/32.png) [@vvsvinu](https://discuss.elastic.co/u/vvsvinu)\
**Post date:** [January 1, 2018, 10:39am UTC](https://discuss.elastic.co/t/new-install-elk-6-1-with-filebeat-6-1-for-tomcat-catalina-logs/113559/4 "2018-01-01T10:39:20Z")

</div>

For tomcat configuration I have edited my filebeat.yml and logstash\_tomcat.conf file as follows. Now Iam receiving the log file in my kibana interface. But while traversing through my log files, it was found that Iam getting "\_grokparsefailure" which is not in the logs. Also in the same log series for another entry Iam getting "\_dateparsefailure". Please find the required configuration files ie. filebeat.yml, logstash\_tomcat.conf, JSON output and the error snippet passed mentioned along.

Please help me if my configurations are incorrect as Iam new to ELK+Filebeat.

############# Filebeat.yml ###############

filebeat.prospectors:

- type: log  
enabled: true  
paths:
  - /usr/local/tomcat/logs/catalina.out  
multiline.pattern: '^[[:space:]]'  
multiline.pattern: '^[[:space:]]+|^Caused by:'  
multiline.negate: false  
multiline.match: after  
document\_type: tomcat\_log  
filebeat.config.modules:  
path: ${path.config}/modules.d/\*.yml  
reload.enabled: false  
setup.template.settings:  
index.number\_of\_shards: 3  
setup.kibana:  
host: "192.168.3.226:5601"  
output.logstash:  
hosts: ["192.168.3.226:5044"]

##########################################

############# logstash\_tomcat.conf #######

input {  
beats {  
type =\> "tomcat\_log"  
host =\> "192.168.3.226"  
port =\> 5044  
}  
}

filter {  
if [type] == "tomcat\_log" {  
grok {

# patterns\_dir =\> "/etc/logstash/patterns"

```
match => ["message", "(?m)%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:severity} %{GREEDYDATA:message}"]
overwrite => ["message"]

```

}  
date {  
match =\> ["timestamp", "yyyy-MM-dd HH:mm:ss,SSS"]  
remove\_field =\> ["timestamp"]  
}

}  
}  
output {  
stdout { codec =\> rubydebug }  
if [type] == "tomcat\_log" {  
elasticsearch {  
manage\_template =\> false  
hosts =\> ["192.168.3.226:9200"]  
}  
}  
}

##########################################  
##################\_grokparsefailure ######

{  
"\_index": "logstash-2018.01.01",  
"\_type": "doc",  
"\_id": "QnYPsWABQTfQ6qM1YymO",  
"\_version": 1,  
"\_score": null,  
"\_source": {  
"source": "/usr/local/tomcat/logs/catalina.out",  
"@version": "1",  
"host": "kibana.ctax.dev",  
"type": "tomcat\_log",  
"beat": {  
"version": "6.1.1",  
"name": "kibana.ctax.dev",  
"hostname": "kibana.ctax.dev"  
},  
"offset": 29553175,  
"message": "Hibernate: insert into gst\_log.gstn\_api\_log (api\_versn, encrpt\_key, reqst\_body, reqst\_date, reqst\_query\_strng, reqst\_type, reqst\_url, reqst\_year\_month, rspns\_sts, rspns\_body, rspns\_ek, rspns\_hmac, reqst\_rspns\_id) VALUES (?, ?, ?::jsonb, ?::timestamptz, ?, ?, ?, ?::int, ?, ?::jsonb, ?, ?, ?::int)",  
"prospector": {  
"type": "log"  
},  
"tags": [  
"beats\_input\_codec\_plain\_applied",  
"\_grokparsefailure"  
],  
"@timestamp": "2018-01-01T09:30:57.951Z"  
},  
"fields": {  
"@timestamp": [  
"2018-01-01T09:30:57.951Z"  
]  
},  
"sort": [  
1514799057951  
]  
}  
##########################################

##################\_dateparsefailure ######  
{  
"\_index": "logstash-2018.01.01",  
"\_type": "doc",  
"\_id": "go8dsWABQTfQ6qM1d8us",  
"\_score": 1,  
"\_source": {  
"source": "/usr/local/tomcat/logs/catalina.out",  
"@version": "1",  
"timestamp": "2017-10-13 16:00:00",  
"beat": {  
"version": "6.1.1",  
"name": "kibana.ctax.dev",  
"hostname": "kibana.ctax.dev"  
},  
"offset": 593030236,  
"message": "TaskUtils$LoggingErrorHandler:95 - Unexpected error occurred in scheduled task.",  
"type": "tomcat\_log",  
"host": "kibana.ctax.dev",  
"severity": "ERROR",  
"prospector": {  
"type": "log"  
},  
"tags": [  
"beats\_input\_codec\_plain\_applied",  
"\_dateparsefailure"  
],  
"@timestamp": "2018-01-01T09:46:25.418Z"  
},  
"fields": {  
"@timestamp": [  
"2018-01-01T09:46:25.418Z"  
]  
}  
}  
##########################################

############ Error-log snippet ###########

Hibernate: insert into gst\_log.gstn\_api\_log (api\_versn, encrpt\_key, reqst\_body, reqst\_date, reqst\_query\_strng, reqst\_type, reqst\_url, reqst\_year\_month, rspns\_sts, rspns\_body, rspns\_ek, rspns\_hmac, reqst\_rspns\_id) VALUES (?, ?, ?::jsonb, ?::timestamptz, ?, ?, ?, ?::int, ?, ?::jsonb, ?, ?, ?::int)  
2017-10-31 16:00:00 ERROR TaskUtils$LoggingErrorHandler:95 - Unexpected error occurred in scheduled task.  
java.lang.Error: Unresolved compilation problem:  
Unhandled exception type ParseException

```
    at nic.kerala.adsm.attendancesystem.configuration.SchedulerConfig.scheduleTaskUsingCronExpression(SchedulerConfig.java:49)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:65)
    at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
    at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:748)

```

Hibernate: select nextval ('gst\_log.gstn\_api\_log\_reqst\_rspns\_id\_seq')  
Hibernate: insert into gst\_log.gstn\_api\_log (api\_versn, encrpt\_key, reqst\_body, reqst\_date, reqst\_query\_strng, reqst\_type, reqst\_url, reqst\_year\_month, rspns\_sts, rspns\_body, rspns\_ek, rspns\_hmac, reqst\_rspns\_id) VALUES (?, ?, ?::jsonb, ?::timestamptz, ?, ?, ?, ?::int, ?, ?::jsonb, ?, ?, ?::int)  
Hibernate: insert into gst\_log.gstn\_api\_log (api\_versn, encrpt\_key, reqst\_body, reqst\_date, reqst\_query\_strng, reqst\_type, reqst\_url, reqst\_year\_month, rspns\_sts, rspns\_body, rspns\_ek, rspns\_hmac, reqst\_rspns\_id) VALUES (?, ?, ?::jsonb, ?::timestamptz, ?, ?, ?, ?::int, ?, ?::jsonb, ?, ?, ?::int)

##########################################

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [January 1, 2018, 2:48pm UTC](https://discuss.elastic.co/t/new-install-elk-6-1-with-filebeat-6-1-for-tomcat-catalina-logs/113559/5 "2018-01-01T14:48:21Z")

</div>

You're barking up the wrong tree with the Filebeat multiline configuration. The logic you're looking for is "unless the line begins with a timestamp, join with the next line". Follow the pattern at [https://www.elastic.co/guide/en/beats/filebeat/current/\_examples\_of\_multiline\_configuration.html#\_timestamps](https://www.elastic.co/guide/en/beats/filebeat/current/_examples_of_multiline_configuration.html#_timestamps).

Once the multiline processing is done correctly you can start worrying about any grok and date parse failures.

---

<div class="post-metadata">

**Author:** ![vvsvinu](https://avatars.discourse-cdn.com/v4/letter/v/a88e57/32.png) [@vvsvinu](https://discuss.elastic.co/u/vvsvinu)\
**Post date:** [January 3, 2018, 11:52am UTC](https://discuss.elastic.co/t/new-install-elk-6-1-with-filebeat-6-1-for-tomcat-catalina-logs/113559/6 "2018-01-03T11:52:40Z")

</div>

Hai,

As per your link description I have updated my filebeat.yml file as follows to read the error-log file based on time. Please help me in clearing with "\_grokparsefailure" and "\_dateparsefailure" error which Iam still facing.

############ filebeat.yml ######################

## filebeat.prospectors:

```
type: log
enabled: true
multiline.match: after
multiline.negate: true
multiline.pattern: '[0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}'
paths:
  - /usr/local/tomcat/logs/catalina.out
fields: {log_type: tomcat1_cat_log}
document_type: tomcat_log

```

filebeat.config.modules:  
path: ${path.config}/modules.d/\*.yml  
reload.enabled: false  
setup.template.settings:  
index.number\_of\_shards: 3  
setup.kibana:  
host: "192.168.3.226:5601"  
output.logstash:  
hosts: ["192.168.3.226:5044"]

################################################

Also, if I have missed anything, please help me correct my filebeat.yml file..

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [January 3, 2018, 7:05pm UTC](https://discuss.elastic.co/t/new-install-elk-6-1-with-filebeat-6-1-for-tomcat-catalina-logs/113559/7 "2018-01-03T19:05:36Z")

</div>

Please use a `stdout { codec => rubydebug }` output and show an example event produced by Logstash, plus your Logstash configuration.

---

<div class="post-metadata">

**Author:** ![vvsvinu](https://avatars.discourse-cdn.com/v4/letter/v/a88e57/32.png) [@vvsvinu](https://discuss.elastic.co/u/vvsvinu)\
**Post date:** [January 4, 2018, 11:07am UTC](https://discuss.elastic.co/t/new-install-elk-6-1-with-filebeat-6-1-for-tomcat-catalina-logs/113559/8 "2018-01-04T11:07:54Z")

</div>

Hai, magnusbaeck Plz find the output... it still contains the "\_dateparsefailure"..

#############################################  
{  
"beat" =\> {  
"hostname" =\> "kibana.ctax.dev",  
"version" =\> "6.1.1",  
"name" =\> "kibana.ctax.dev"  
},  
"source" =\> "/usr/local/tomcat/logs/catalina.out",  
"type" =\> "tomcat\_log",  
"@timestamp" =\> 2018-01-04T10:57:19.283Z,  
"severity" =\> "ERROR",  
"message" =\> "LocalDataSourceJobStore:2867 - Error retrieving job, setting trigger state to ERROR.\norg.quartz.JobPersistenceException: Couldn't retrieve job because a required class was not found: nic.kerala.gst.scheduler.jobs.ledger.CashLedgerJob [See nested exception: java.lang.ClassNotFoundException: nic.kerala.gst.scheduler.jobs.ledger.CashLedgerJob]\n\tat org.quartz.impl.jdbcjobstore.JobStoreSupport.retrieveJob(JobStoreSupport.java:1393)\n\tat org.quartz.impl.jdbcjobstore.JobStoreSupport.acquireNextTrigger(JobStoreSupport.java:2864)\n\tat org.quartz.impl.jdbcjobstore.JobStoreSupport$41.execute(JobStoreSupport.java:2805)\n\tat org.quartz.impl.jdbcjobstore.JobStoreSupport$41.execute(JobStoreSupport.java:2803)\n\tat org.quartz.impl.jdbcjobstore.JobStoreSupport.executeInNonManagedTXLock(JobStoreSupport.java:3849)\n\tat org.quartz.impl.jdbcjobstore.JobStoreSupport.acquireNextTriggers(JobStoreSupport.java:2802)\n\tat org.quartz.core.QuartzSchedulerThread.run(QuartzSchedulerThread.java:287)\nCaused by: java.lang.ClassNotFoundException: nic.kerala.gst.scheduler.jobs.ledger.CashLedgerJob\n\tat org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1285)\n\tat org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1119)\n\tat org.springframework.scheduling.quartz.ResourceLoaderClassLoadHelper.loadClass(ResourceLoaderClassLoadHelper.java:76)\n\tat org.springframework.scheduling.quartz.ResourceLoaderClassLoadHelper.loadClass(ResourceLoaderClassLoadHelper.java:81)\n\tat org.quartz.impl.jdbcjobstore.StdJDBCDelegate.selectJobDetail(StdJDBCDelegate.java:852)\n\tat org.quartz.impl.jdbcjobstore.JobStoreSupport.retrieveJob(JobStoreSupport.java:1390)\n\t... 6 more",  
"offset" =\> 2192,  
"host" =\> "kibana.ctax.dev",  
"tags" =\> [  
[0] "beats\_input\_codec\_plain\_applied",  
[1] "\_dateparsefailure"  
],  
"timestamp" =\> "2017-11-02 23:59:50",  
"prospector" =\> {  
"type" =\> "log"  
},  
"fields" =\> {  
"log\_type" =\> "tomcat1\_cat\_log"  
},  
"@version" =\> "1"  
}

#############################################

############# logstash.yml #################

path.data: /var/lib/logstash  
path.config: /etc/logstash/conf.d/\*.conf  
path.logs: /var/log/logstash

############################################

############# conf.d/logstash\_tomcat.conf ####

input {  
beats {  
type =\> "tomcat\_log"  
host =\> "192.168.3.226"  
port =\> 5044  
}  
}

filter {  
if [type] == "tomcat\_log" {  
grok {  
patterns\_dir =\> "/etc/logstash/patterns"  
match =\> ["message", "(?m)%{TIMESTAMP\_ISO8601:timestamp} %{LOGLEVEL:severity} %{GREEDYDATA:message}"]  
overwrite =\> ["message"]  
}  
date {  
match =\> ["timestamp", "yyyy-MM-dd HH:mm:ss,SSS"]  
remove\_field =\> ["timestamp"]  
}

}  
}  
output {  
stdout { codec =\> rubydebug }  
if [type] == "tomcat\_log" {  
elasticsearch {  
manage\_template =\> false  
hosts =\> ["192.168.3.226:9200"]  
}  
}  
}

############################################

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [January 4, 2018, 2:30pm UTC](https://discuss.elastic.co/t/new-install-elk-6-1-with-filebeat-6-1-for-tomcat-catalina-logs/113559/9 "2018-01-04T14:30:11Z")

</div>

Your date pattern doesn't match the actual timestamp in the `timestamp` field (drop the milliseconds).

---

<div class="post-metadata">

**Author:** ![vvsvinu](https://avatars.discourse-cdn.com/v4/letter/v/a88e57/32.png) [@vvsvinu](https://discuss.elastic.co/u/vvsvinu)\
**Post date:** [January 8, 2018, 4:54am UTC](https://discuss.elastic.co/t/new-install-elk-6-1-with-filebeat-6-1-for-tomcat-catalina-logs/113559/10 "2018-01-08T04:54:10Z")

</div>

Hai, magnusbaeck, Thanks for pinpointing timestamp format. This had solved by "\_dateparsefailure" issue.

But while viewing the logs via Kibana UI, it was noticed that the logs are getting scrambled with respect to the log file. Please find my catalina.out image and Kibana UI enclosed along.

Please refer the lines mentioned belows in both the images.

ROOT has finished in 321 ms  
docs has finished in 9 ms  
examples has finished in 186 ms  
host-manager has finished in 13 ms  
manager has finished in 10 ms

Screen-shot Kibana UI

 ![Screenshot at 2018-01-08 10-04-08](https://us1.discourse-cdn.com/elastic/original/3X/b/3/b38361593a2c1e134ce6608be3ee0244c986ef36.png)

Screen-shot Catalina.out

 ![Screenshot at 2018-01-08 10-00-28](https://us1.discourse-cdn.com/elastic/original/3X/d/8/d829ef28f92bd974a68821c664b2cdb775b4d623.png)

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [January 8, 2018, 8:24am UTC](https://discuss.elastic.co/t/new-install-elk-6-1-with-filebeat-6-1-for-tomcat-catalina-logs/113559/11 "2018-01-08T08:24:02Z")

</div>

Kibana doesn't preserve the original order. It orders events based on the timestamp, and with only second resolution that's not going to give you the right order.

---

<div class="post-metadata">

**Author:** ![vvsvinu](https://avatars.discourse-cdn.com/v4/letter/v/a88e57/32.png) [@vvsvinu](https://discuss.elastic.co/u/vvsvinu)\
**Post date:** [January 16, 2018, 9:56am UTC](https://discuss.elastic.co/t/new-install-elk-6-1-with-filebeat-6-1-for-tomcat-catalina-logs/113559/12 "2018-01-16T09:56:31Z")

</div>

Hai, magnusbaeck ... Now I have been reciveing the grokparsefailure error mentioned in my earlier post. Plz find the output that contains the "\_grokparsefailure"..

############################################# ERROR SNIPPET #############

"@timestamp" =\> 2018-01-16T04:59:22.000Z,  
"type" =\> "tomcat\_log",  
"severity" =\> "ERROR"  
}  
{  
"message" =\> " Source field value: nic.kerala.gst.registration.entity.gstn.Gstp@22411420b4[professionalAddressDetails=nic.kerala.gst.registration.entity.gstn.Address@30sacee48[addressId=1289041,bldgNum=test house wwwwww po,floorNum=,bldgName=,strt=sswaruvikuzhy,locty=anwwickadu,dst=KLKWWOT,stateCode=3112,pinCode=68622503,lat=,lon=,adressType=PA,aplnType=RTTR1,entyType=RTTR1,cntry=,cntryCode=,documents=[nic.kerala.gst.registration.entity.gstn.DocUpld@61ac2b85d],resubmittedDocuments=],enrollmentDetails=nic.kerala.gst.registration.entity.gstn.GstpEnrmtDtls@10f28eb5d,graduationDetails=nic.kerala.gst.registration.entity.gstn.GstpGrdtnDtls@49d289d53,applicantDetails=nic.kerala.gst.registration.entity.gstn.GstpAplntsDtls@505d6969,oldGstPractnrDetails=,queryClrfnData=,scnQueries=,modfctnRegnApln=,addnlInfo=,draftId=3404234,documents=,arnDtlsId=405608,aplnType=RTTR1,arn=AA3201180050423,createdDate=2018-01-12 21:00:16.497735,modifiedDate=,regnDtls=nic.kerala.gst.registration.entity.gstn.RegnDtls@3f0ffd52[arnDtlsId=405608,aplnType=RTTR1,aplnState=PFV,modfdDate=2018-01-12,dueDate=,email=ert345@gmail.com,mob=8222210651,date=,pan=FEPP22227L,pt=,legalNameBsns=,atzdSgnryName=,submitDate=,dsrctCode=KLKOT,stateCode=3222,aplnSts=,gstPrctnrName=xxx wwww,panOrTan=,cpin=,provsnlId=],decln=nic.kerala.gst.registration.entity.gstn.Decln@72956d9c[arnDtlsId=405608,declnPlace=aruwwvikuzhy,declnDate=2018-01-12,signType=EVC,asDescn=,asNum=,declnName=,declnText=,verified=true,applicantName=rrrrrrrr,pan=FEPPS0rr937L],origHash=b2brrrrrdef7f2af98525fed8b4c209e1rr64edc4e7feadaa1rb515a1158bed9,lastActionDate=2018-01-28,isMigrated=,securityToken=32184444400003599TRN,gstin=,current=true,alertDtlsId=]\n Dest parent class: nic.kerala.gst.registration.vo.GstpEnrmtDrrrrtlsVO\n Dest field name: gstp\n Dest field type: nic.kerala.gst.registration.entity.gstn.Gstp\norg.hibernate.LazyInitializationException: failed to lazily initialize a collection of role: nic.kerala.gst.registration.entity.gstn.GstpAplyyntsDtls.resubmittedPhotos, could not initialize proxy - no Session\n\tat org.hibernate.collection.internal.AbstractPersistentCollection.throwLazyInitializationException(AbstractPersistentCollection.java:582)\n\tat org.hibernate.collection.internal.AbstractPersistentCollection.withTemporarySessionIfNeeded(AbstractPersistentCollection.java:201)\n\tat org.hibernate.collection.internal.AbstractPersistentCollection.readSize(AbstractPersistentCollection.java:145)\n\tat org.hibernate.collection.internal.PersistentBag.size(PersistentBag.java:261)\n\tat org.dozer.MappingProcessor.prepareDestinationList(MappingProcessor.java:837)\n\tat org.dozer.MappingProcessor.addOrUpdateToList(MappingProcessor.java:762)\n\tat org.dozer.MappingProcessor.addOrUpdateToList(MappingProcessor.java:850)\n\tat org.dozer.MappingProcessor.mapListToList(MappingProcessor.java:686)\n\tat org.dozer.MappingProcessor.mapCollection(MappingProcessor.java:541)\n\tat org.dozer.MappingProcessor.mapOrRecurseObject(MappingProcessor.java:434)\n\tat org.dozer.MappingProcessor.mapFromFieldMap(MappingProcessor.java:342)\n\tat org.dozer.MappingProcessor.mapField(MappingProcessor.java:288)\n\tat org.dozer.MappingProcessor.map(MappingProcessor.java:248)\n\tat org.dozer.MappingProcessor.map(MappingProcessor.java:197)\n\tat org.dozer.MappingProcessor.mapCustomObject(MappingProcessor.java:495)\n\tat org.dozer.MappingProcessor.mapOrRecurseObject(MappingProcessor.java:446)\n\tat org.dozer.MappingProcessor.mapFromFieldMap(MappingProcessor.java:342)\n\tat org.dozer.MappingProcessor.mapField(MappingProcessor.java:288)\n\tat org.dozer.MappingProcessor.map(MappingProcessor.java:248)\n\tat org.dozer.MappingProcessor.map(MappingProcessor.java:197)\n\tat org.dozer.MappingProcessor.mapCustomObject(MappingProcessor.java:495)\n\tat org.dozer.MappingProcessor.mapOrRecurseObject(MappingProcessor.java:446)\n\tat org.dozer.MappingProcessor.mapFromFieldMap(MappingProcessor.java:342)\n\tat org.dozer.MappingProcessor.mapField(MappingProcessor.java:288)\n\tat org.dozer.MappingProcessor.map(MappingProcessor.java:248)\n\tat org.dozer.MappingProcessor.map(MappingProcessor.java:197)\n\tat org.dozer.MappingProcessor.mapCustomObject(MappingProcessor.java:495)\n\tat o org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)\n\tat org.apache.coyote.ajp.AjpProcessor.service(AjpProcessor.java:486)\n\tat org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)\n\tat org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:861)\n\tat org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1455)\n\tat org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)\n\tat org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)\n\tat java.lang.Thread.run(Thread.java:748)",  
"fields" =\> {  
"log\_type" =\> "tomcat1\_cat\_log"  
},  
"prospector" =\> {  
"type" =\> "log"  
},  
"tags" =\> [  
[0] "beats\_input\_codec\_plain\_applied",  
[1] "\_grokparsefailure"  
],  
"@version" =\> "1",  
"host" =\> "kibana.ctax.dev",  
"offset" =\> 967530,  
"beat" =\> {  
"name" =\> "kibana.ctax.dev",  
"hostname" =\> "kibana.ctax.dev",  
"version" =\> "6.1.1"  
},  
"source" =\> "/usr/local/tomcat/logs/catalina.out",  
"@timestamp" =\> 2018-01-16T09:22:17.381Z,  
"type" =\> "tomcat\_log"  
}

#############################################

\*\*\* The error enclosed along has been trimmed. The Complete error report has been enclosed along for reference

The logstash.yml, logstash-tomcat.conf and filebeat.yml configurations remains the same as above

Complete error-log part-1

 ![Screenshot at 2018-01-16 15-14-05](https://us1.discourse-cdn.com/elastic/original/3X/a/7/a79c838883d0ef8153a1553bf239002ebb8671a3.png)

Complete error-log part-2

 ![Screenshot at 2018-01-16 15-13-45](https://us1.discourse-cdn.com/elastic/original/3X/9/7/97d34fa7b251f9dc52896814318b0b43d24b968c.png)

Complete error-log part-3

 ![Screenshot at 2018-01-16 15-14-14](https://us1.discourse-cdn.com/elastic/original/3X/2/9/29cfa4350e73085edc79bc58f6de5c482d4a5157.png)

---

<div class="post-metadata">

**Author:** ![vvsvinu](https://avatars.discourse-cdn.com/v4/letter/v/a88e57/32.png) [@vvsvinu](https://discuss.elastic.co/u/vvsvinu)\
**Post date:** [January 16, 2018, 9:59am UTC](https://discuss.elastic.co/t/new-install-elk-6-1-with-filebeat-6-1-for-tomcat-catalina-logs/113559/13 "2018-01-16T09:59:07Z")

</div>

Please find the modified grok-patterns file snippet shown below for traversing the tomcat catalina.out  
log. I have modified the default grok-patterns file by adding additional Java Logs section to traverse tomcat/java stack trace log as shown above.

########## /etc/logstash/patterns/grok-patterns ##########  
......

# Log formats

SYSLOGBASE %{SYSLOGTIMESTAMP:timestamp} (?:%{SYSLOGFACILITY} )?%{SYSLOGHOST:logsource} %{SYSLOGPROG}:

# Log Levels

LOGLEVEL ([Aa]lert|ALERT|[Tt]race|TRACE|[Dd]ebug|DEBUG|[Nn]otice|NOTICE|[Ii]nfo|INFO|[Ww]arn?(?:ing)?|WARN?(?:ING)?|[Ee]rr?(?:or)?|ERR?(?:OR)?|[Cc]rit?(?:ical)?|CRIT?(?:ICAL)?|[Ff]atal|FATAL|[Ss]evere|SEVERE|EMERG(?:ENCY)?|[Ee]merg(?:ency)?)

**####### Modified by me ############**

# Java Logs

JAVATHREAD (?:[A-Z]{2}-Processor[\d]+)  
JAVACLASS (?:[a-zA-Z0-9-]+.)+[A-Za-z0-9$]+  
JAVAFILE (?:[A-Za-z0-9\_.-]+)  
JAVASTACKTRACEPART at %{JAVACLASS:class}.%{WORD:method}(%{JAVAFILE:file}:%{NUMBER:line})  
JAVALOGMESSAGE (.\*)

# MMM dd, yyyy HH:mm:ss eg: Jan 9, 2014 7:13:13 AM

CATALINA\_DATESTAMP %{MONTH} %{MONTHDAY}, 20%{YEAR} %{HOUR}:?%{MINUTE}(?::?%{SECOND}) (?:AM|PM)

# yyyy-MM-dd HH:mm:ss,SSS ZZZ eg: 2014-01-09 17:32:25,527 -0800

TOMCAT\_DATESTAMP 20%{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{HOUR}:?%{MINUTE}(?::?%{SECOND}) %{ISO8601\_TIMEZONE}  
CATALINALOG %{CATALINA\_DATESTAMP:timestamp} %{JAVACLASS:class} %{JAVALOGMESSAGE:logmessage}

# 2014-01-09 20:03:28,269 -0800 | ERROR | com.example.service.ExampleService - something compeletely unexpected happened...

TOMCATLOG %{TOMCAT\_DATESTAMP:timestamp} | %{LOGLEVEL:level} | %{JAVACLASS:class} - %{JAVALOGMESSAGE:logmessage}

#################################################

---

<div class="post-metadata">

**Author:** ![vvsvinu](https://avatars.discourse-cdn.com/v4/letter/v/a88e57/32.png) [@vvsvinu](https://discuss.elastic.co/u/vvsvinu)\
**Post date:** [January 20, 2018, 10:56am UTC](https://discuss.elastic.co/t/new-install-elk-6-1-with-filebeat-6-1-for-tomcat-catalina-logs/113559/14 "2018-01-20T10:56:27Z")

</div>

any update ..please

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 26, 2018, 9:40am UTC](https://discuss.elastic.co/t/new-install-elk-6-1-with-filebeat-6-1-for-tomcat-catalina-logs/113559/15 "2018-01-26T09:40:15Z")

</div>

This topic was automatically closed after 28 days. New replies are no longer allowed.