# Failed Parsing Date and MulitLine tomcat log

**URL:** <https://discuss.elastic.co/t/failed-parsing-date-and-mulitline-tomcat-log/41537>\
**Category:** Logstash\
**Created:** [February 11, 2016, 6:09pm UTC](https://discuss.elastic.co/t/failed-parsing-date-and-mulitline-tomcat-log/41537 "2016-02-11T18:09:01Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![bilguuna](https://avatars.discourse-cdn.com/v4/letter/b/977dab/32.png) [@bilguuna](https://discuss.elastic.co/u/bilguuna)\
**Post date:** [February 11, 2016, 6:09pm UTC](https://discuss.elastic.co/t/failed-parsing-date-and-mulitline-tomcat-log/41537/1 "2016-02-11T18:09:01Z")

</div>

Hello, I've been trying to read tomcat log and make some analysis such as number of errors and exceptions raised etc. Here is my conf:[code]  
input {  
file {  
path =\> "/opt/logstash/localhost.log"  
start\_position =\> beginning  
sincedb\_path =\> "/var/log/logstash/null"  
}  
}  
filter {  
grok {  
match =\> { "message" =\> "(?%{MONTHDAY}-%{MONTH}-%{YEAR} %{HOUR}:%{MINUTE}:%{SECOND}) %{LOGLEVEL:logLevel} %{GREEDYDATA:message}" }  
}  
mutate {  
add\_field =\> { "logLevel" =\> "%{logLevel}" }  
}  
date {  
locale =\> "en"  
match =\> ["logdate", "dd-MMM-yyyy HH:mm:ss", "ISO8601"]  
target =\> "@timestamp"  
add\_tag =\> ["tmatch"]  
}  
}  
output {  
elasticsearch {  
hosts =\> ["localhost:9200"]  
}  
stdout { codec =\> rubydebug }  
}

[/code]  
And below is some of the lines in my log file: `
09-Jan-2016 18:30:38.722 INFO [localhost-startStop-1] org.apache.catalina.core.ApplicationContext.log No Spring WebApplicationInitializer types detected on classpath
09-Jan-2016 18:30:38.796 INFO [localhost-startStop-1] org.apache.catalina.core.ApplicationContext.log Initializing log4j from [C:\tomcat\apache-tomcat-8.0.26\temp\0-contact-statecollab-ws-15.12-SNAPSHOT-unknown-20151230-1152\WEB-INF\log4j.properties`

I have few of questions/problems:

1. I got following exception:  
`Failed parsing date from field {:field=>"logdate", :value=>"19-Jan-2016 18:30:38.722", :exception=>"Invalid format: \"19-Jan-2016 18:30:38.722\" is malformed at \"Jan-2016 18:30:38.722\"", :config_parsers=>"dd-MMM-yyyy HH:mm:ss,ISO8601", :config_locale=>"en", :level=>:warn}`
2. How should I use logdate instead of @timestamp
3. How should I deal with multi line tomcat log?

Any suggestion would be appreciated. Thanks

---

<div class="post-metadata">

**Author:** ![bilguuna](https://avatars.discourse-cdn.com/v4/letter/b/977dab/32.png) [@bilguuna](https://discuss.elastic.co/u/bilguuna)\
**Post date:** [February 11, 2016, 7:51pm UTC](https://discuss.elastic.co/t/failed-parsing-date-and-mulitline-tomcat-log/41537/2 "2016-02-11T19:51:58Z")

</div>

I found the problem with date, all I had to do was that add .SSS(second precision) to the date filter. So that it looks like:

```auto
date {
        locale => "en"
        match => ["logdate", "dd-MMM-yyyy HH:mm:ss.SSS", "ISO8601"]
    }
```

Now it gets timestamp correctly. But how should I deal with multiline error msg such as:

```auto
09-Jan-2016 18:43:29.405 SEVERE [http-apr-8080-exec-9] org.apache.catalina.core.StandardContext.listenerStart Exception sending context initialized event to listener instance of class org.springframework.web.context.ContextLoaderListener
 org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'exports-ws': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire field: protected gov.hhs.acf.ohs.hses.model.exports.service.ExportsService gov.hhs.acf.ohs.hses.services.exports.ExportsWebService.exportsService; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type [gov.hhs.acf.ohs.hses.model.exports.service.ExportsService] found for dependency: expected at least 1 bean which qualifies as autowire candidate for this dependency. Dependency annotations: {@org.springframework.beans.factory.annotation.Autowired(required=true), @org.springframework.beans.factory.annotation.Qualifier(value=exportsService)}
Related cause: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'gisDataSource': Invocation of init method failed; nested exception is javax.naming.NameNotFoundException: Name [hhhh_service] is not bound in this Context. Unable to find [hses_service].
Related cause: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'contactService' defined in class path resource [contactContext.xml]: Cannot resolve reference to bean 'contactPgTransactionManager' while setting bean property 'transactionManager'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'contactPgTransactionManager' defined in class path resource [contactDaoPgContext.xml]: Cannot resolve reference to bean 'contactPgSessionFactory' while setting bean property 'sessionFactory'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'contactPgSessionFactory' defined in class path resource [contactDaoPgContext.xml]: Cannot resolve reference to bean 'gisDataSource' while setting bean property 'dataSource'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'gisDataSource': Invocation of init method failed; nested exception is javax.naming.NameNotFoundException: Name [hhhh_service] is not bound in this Context. Unable to find [hses_service].
```

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 11, 2016, 8:38pm UTC](https://discuss.elastic.co/t/failed-parsing-date-and-mulitline-tomcat-log/41537/3 "2016-02-11T20:38:29Z")

</div>

You need to use the [multiline codec](https://www.elastic.co/guide/en/logstash/current/plugins-codecs-multiline.html). Its documentation contains a few examples of how it can be used.

---

<div class="post-metadata">

**Author:** ![bilguuna](https://avatars.discourse-cdn.com/v4/letter/b/977dab/32.png) [@bilguuna](https://discuss.elastic.co/u/bilguuna)\
**Post date:** [February 11, 2016, 10:12pm UTC](https://discuss.elastic.co/t/failed-parsing-date-and-mulitline-tomcat-log/41537/4 "2016-02-11T22:12:28Z")

</div>

Thank you Magnus, I added below lines to my config:

```auto
codec => multiline {
            pattern => "(dd-MMM-yyyy HH:mm:ss.SSS^\s*)"
            what => "next"
            }
        }
```

Basically, Each line should be started with date, if not, a line should be considered as continues of previous line. But it still count each line as a new line. Did I miss anything?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 12, 2016, 6:34am UTC](https://discuss.elastic.co/t/failed-parsing-date-and-mulitline-tomcat-log/41537/5 "2016-02-12T06:34:11Z")

</div>

Two problems:

- Your pattern is not a regular expression. It looks like the pattern used for a date filter. Use your grok expression as a starting point instead.
- The logic is flawed since it'll only work for sequences of two lines. If you have a sequence of three lines that should be joined, the second line won't begin with a timestamp so it won't be joined with the following line. Instead, turn the logic around with `negate => true` and use `what => "previous"`, expressing "if the line _doesn't_ begin with a timestamp, join it with the _preceding_ line".

---

<div class="post-metadata">

**Author:** ![bilguuna](https://avatars.discourse-cdn.com/v4/letter/b/977dab/32.png) [@bilguuna](https://discuss.elastic.co/u/bilguuna)\
**Post date:** [February 12, 2016, 4:01pm UTC](https://discuss.elastic.co/t/failed-parsing-date-and-mulitline-tomcat-log/41537/6 "2016-02-12T16:01:49Z")

</div>

Thank you Magnus, You explained it way better than the documentation. I replaced a pattern with grok expression and set max\_lines since it's more than 500 rows. Now it's working as expected.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:11am UTC](https://discuss.elastic.co/t/failed-parsing-date-and-mulitline-tomcat-log/41537/7 "2017-07-06T05:11:46Z")

</div>