# Parsing multiline Tomcat logs

**URL:** https://discuss.elastic.co/t/parsing-multiline-tomcat-logs/178874
**Category:** Logstash
**Created:** [April 29, 2019, 9:05am UTC](https://discuss.elastic.co/t/parsing-multiline-tomcat-logs/178874 "2019-04-29T09:05:32Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![agentn](https://avatars.discourse-cdn.com/v4/letter/a/e274bd/32.png) [@agentn](https://discuss.elastic.co/u/agentn)
#### Post date: [April 29, 2019, 9:05am UTC](https://discuss.elastic.co/t/parsing-multiline-tomcat-logs/178874/1 "2019-04-29T09:05:32Z")

</div>

Hi,  
I'm trying to parse some Tomcat logs with logstash, here is an example of data :

> **Summary**
>
> ```
> 25-04-19 10:16:00 WARN [BasicResourcePool] com.mchange.v2.resourcepool.BasicResourcePool@74f26dc -- Acquisition Attempt Failedclear! Clearing pending acquires. While trying to acquire a needed new resource, we failed to succeed more than the maximum number of allowed acquisition attempts (30).
> 26-04-19 11:38:00 WARN [BasicResourcePool] com.mchange.v2.resourcepool.BasicResourcePool@74f26dc -- Acquisition Attempt Failedclear! Clearing pending acquires. While trying to acquire a needed new resource, we failed to succeed more than the maximum number of allowed acquisition attempts (30). Last acquisition attempt exception: TEST STACKTRACE BIS
> org.postgresql.util.PSQLException: Connection to ii refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
> at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:265)
> at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
> at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:194)
> at org.postgresql.Driver.makeConnection(Driver.java:431)
> at org.postgresql.Driver.connect(Driver.java:247)
> at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:175)
> at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:220)
> at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:206)
> at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPoolPooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:203)
> at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1138)
> at com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1125)
> at com.mchange.v2.resourcepool.BasicResourcePool.access00(BasicResourcePool.java:44)
> at com.mchange.v2.resourcepool.BasicResourcePool.run(BasicResourcePool.java:1870)
> at com.mchange.v2.async.ThreadPoolAsynchronousRunner .run(ThreadPoolAsynchronousRunner.java:696)
> Caused by: java.net.ConnectException: Network is unreachable (connect failed)
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.net.Socket.connect(Socket.java:589)
> at org.postgresql.core.PGStream.<init>(PGStream.java:62)
> at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:144)
> ... 13 more
> 
> ```

I have seen on others topics that multilines plugin and patterns are usefull :

Pattern file :

> **Summary**
>
> `JAVA_TRACE (^.+Exception: .+)|(^\s+at .+)|(^\s+... \d+ more)|(^\s*Caused by:.+)`

Logstash config file :

> **Summary**
>
> ```
> input {
> tcp{
> port => 12201
> }
> udp{
> port => 12201
> }
> 
> ```
> 
> stdin{  
> codec =\> multiline {  
> pattern =\> "^%{DATE\_EU}"  
> negate =\> "true"  
> what =\> "previous"  
> }  
> }
> 
> ```
> }
> 
> filter{
> if [type]== "tomcat" {
> grok{
> patterns_dir => ["/conf/patterns"]
> match => {message => ["(^%{DATE_EU} %{TIME} %{LOGLEVEL} %{GREEDYDATA}) | (^%{JAVA_TRACE}) "]}
> tag_on_failure => ["failed"]
> }
> 
> }
> 
> }
> 
> output{
> elasticsearch{
> hosts => ["elasticsearch:9200"]
> index => "test"
> }
> }
> 
> ```

I created a grok pattern with a multiline codec, my logs seem to be parsed by the grok, but the multiIine is not working. I suppose that I missed something but i don't know what ... does anyone have an idea ?

Thx for your help.

---

<div class="post-metadata">

### Author: ![agentn](https://avatars.discourse-cdn.com/v4/letter/a/e274bd/32.png) [@agentn](https://discuss.elastic.co/u/agentn)
#### Post date: [May 9, 2019, 9:57am UTC](https://discuss.elastic.co/t/parsing-multiline-tomcat-logs/178874/2 "2019-05-09T09:57:16Z")

</div>

Nobody ? ☹

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 9, 2019, 1:38pm UTC](https://discuss.elastic.co/t/parsing-multiline-tomcat-logs/178874/3 "2019-05-09T13:38:16Z")

</div>

I would expect that multiline codec to result in

```
   "message" => "25-04-19 10:16:00 WARN [BasicResourcePool] com.mchange.v2.resourcepool.BasicResourcePool@74f26dc -- Acquisition Attempt Failedclear! Clearing pending acquires. While trying to acquire a needed new resource, we failed to succeed more than the maximum number of allowed acquisition attempts (30)."

```

for that input. The remainder of the file is being accumulated in another event and waiting for another line that matches %{DATE\_EU} to trigger it being flushed.

You could use the auto\_flush\_interval to flush the event if no such line occurs after a certain number of seconds.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 6, 2019, 1:38pm UTC](https://discuss.elastic.co/t/parsing-multiline-tomcat-logs/178874/4 "2019-06-06T13:38:18Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
