10019531311487253188
33AFED9A8EDFA7BF36E1A116BE52929C:/RCOM_PREPAID
CGRXMA07/10.132.41.87
RCOM_CF_198_GSM_PREPAID
MO
9024019000
NA
16/02/2017 19:23:23
```
**my file beat configuration is**
```
filebeat.prospectors:
- input_type: log
paths:
- C:\ES-Apache\logs\logs_16 feb -2017_prepaid\*.log*
document_type: prepaid_logs
output.logstash:
hosts: ["127.0.0.1:5043"]
# Number of workers per Logstash host. default 1
worker: 4
```
**this is my logstash configuration:**
```
input {
beats {
host => "127.0.0.1"
port => "5043"
}
}
filter {
if [type] == "prepaid_logs" {
if "_grokparsefailure" in [tags] {
drop { }
}
grok {
match => { "message" => "(?:m) %{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:logLevel} - %{POSTFIX_SESSIONID:sessionId}:/%{GREEDYDATA:applicationName} : %{GREEDYDATA:messageText}" }
}
date {
match => ["timestamp", "dd/mm/yyyy:HH:mm:ss:SSS"]
}
}
}
output {
stdout { codec => rubydebug }
if "_grokparsefailure" not in [tags] {
elasticsearch {
hosts => ["127.0.0.1:9200"]
manage_template => false
}
}
}
```
please suggest me right way to solve it. thanks
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [April 25, 2017, 8:25am UTC](https://discuss.elastic.co/t/how-to-parse-tomcat-application-logs/83515/2 "2017-04-25T08:25:44Z")
You'll want a multiline configuration on the Filebeat side to join multiline events, so I'd start in that end.
> i apply some grok patterns but it does'nt work
What does "doesn't work" mean?
---
### Author:  [@yash\_mangla](https://discuss.elastic.co/u/yash_mangla)
#### Post date: [April 25, 2017, 8:31am UTC](https://discuss.elastic.co/t/how-to-parse-tomcat-application-logs/83515/3 "2017-04-25T08:31:23Z")
As you can watch in my configuration
```
grok {
match => { "message" => "(?:m) %{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:logLevel} - %{POSTFIX_SESSIONID:sessionId}:/%{GREEDYDATA:applicationName} : %{GREEDYDATA:messageText}" }
}
```
i applied a grok pattern on message. and i expected to it will create a json field as i want to parse my message.
but it is showing **\_grokparsefailure** error
```
{
"@timestamp" => 2017-04-25T08:27:22.761Z,
"offset" => 2609320,
"@version" => "1",
"input_type" => "log",
"beat" => {
"hostname" => "MANGLA1",
"name" => "MANGLA1",
"version" => "5.3.0"
},
"host" => "MANGLA1",
"source" => "C:\\ES-Apache\\logs\\logs_16 feb -2017_prepaid\\trace.log",
"message" => "16/02/2017 19:21:56:705 INFO - 94C621C85562E801F775D42BA3B4C897:/RCOM_PREPAID : Using SCESession 94C621C85562E801F775D42BA3B4C897:/RCOM_PR
EPAID servlet : LinkDown",
"type" => "prepaid_logs",
"tags" => [
[0] "beats_input_codec_plain_applied",
[1] "_grokparsefailure"
]
}
```
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [April 25, 2017, 8:35am UTC](https://discuss.elastic.co/t/how-to-parse-tomcat-application-logs/83515/4 "2017-04-25T08:35:53Z")
You can't use TIMESTAMP\_ISO8601 because your timestamp is not in ISO8601 format. I think DATE\_EU or DATE\_US should work.
Build your grok expression gradually. Use the grok constructor web site for pointers.
---
### Author:  [@yash\_mangla](https://discuss.elastic.co/u/yash_mangla)
#### Post date: [April 26, 2017, 7:55am UTC](https://discuss.elastic.co/t/how-to-parse-tomcat-application-logs/83515/5 "2017-04-26T07:55:24Z")
Hi @magnusbaeck as you point out me on timestamp i changes this in my pattern. and made a multiline check.
where if my line containing timestamp in beginning it will treated as 1 event and if not begin with timestamp then it is part of previous line.
**i apply these multiline filter settings.**
```
codec => multiline {
patterns_dir => "./logstash/patterns"
pattern => "^%{DATESTAMP}"
negate => true
what => "previous"
}
```
**and this grok pattern to parsing my data.**
```
grok {
match => { "message" => "%{DATESTAMP:timestamp} (?(?:DEBUG|FATAL|ERROR| WARN| INFO)) - (?(%{WORD})):\/(?(%{USERNAME})) : %{GREEDYDATA:error_message}" }
}
date {
match => ["timestamp", "dd/mm/yyyy:HH:mm:ss:SSS"]
}
mutate {
"remove_field" => ["beat", "host", "offset", "@version"]
}
```
It's successfully parsed fields but in tags it showing **\_dateparsefailure** error and i don't have any clue why it's happening.
**output is:**
```
{
"error_message" => "Particular Property File Path :D:\\Program Files\\Apache Software Foundation\\Tomcat 7.0\\webapps\\RCOM\\Prepaid\\RCOM_CF_198_CDMA_PREPAID\\dmProperties\\english\\linkDown.properties",
"input_type" => "log",
"log_level" => "DEBUG",
"session_id" => "40B04527C4894A8F4AA130D619B575A5",
"source" => "C:\\ES-Apache\\logs\\logs_16 feb -2017_prepaid\\trace.log",
"message" => "16/02/2017 19:23:29:276 DEBUG - 40B04527C4894A8F4AA130D619B575A5:/RCOM_PREPAID : Particular Property File Path :D:\\Program Files\\Apache Software Foundation\\Tomcat 7.0\\webapps\\RCOM\\Prepaid\\RCOM_CF_198_CDMA_PREPAID\\dmProperties\\english\\linkDown.properties",
"type" => "prepaid_logs",
"tags" => [
[0] "beats_input_codec_multiline_applied",
[1] "_dateparsefailure"
],
"app_name" => "RCOM_PREPAID",
"@timestamp" => 2017-04-26T07:35:58.813Z,
"timestamp" => "16/02/2017 19:23:29:276"
}
```
you can also test this same pattern on [click here](http://grokconstructor.appspot.com/do/match#result)
now i need help to resolve **\_dateparsefailure** why this is happening.
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [April 26, 2017, 9:16am UTC](https://discuss.elastic.co/t/how-to-parse-tomcat-application-logs/83515/6 "2017-04-26T09:16:10Z")
Use the multiline codec on the Filebeat side, not in Logstash.
There are two reasons why your date filter doesn't work:
- Your timestamp doesn't have a colon between the date and the time.
- You have two occurrences of "mm" in the pattern. Only one of them can be right.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 24, 2017, 9:20am UTC](https://discuss.elastic.co/t/how-to-parse-tomcat-application-logs/83515/7 "2017-05-24T09:20:16Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.