# 2 different time
**URL:** https://discuss.elastic.co/t/2-different-time/266640
**Category:** Logstash
**Created:** [March 9, 2021, 6:56am UTC](https://discuss.elastic.co/t/2-different-time/266640 "2021-03-09T06:56:59Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@111435](https://discuss.elastic.co/u/111435)
#### Post date: [March 9, 2021, 6:56am UTC](https://discuss.elastic.co/t/2-different-time/266640/1 "2021-03-09T06:56:59Z")
Hello guys!
Please, I need your advise.
From one client server to logstash comming 2 different logs with different time template:
1. [2021-02-24 00:00:06,335] [[ACTIVE] ExecuteThread: '121' for queue: 'weblogic.kernel.Default (self-tuning)'] ERROR EntityDataSourceRepository:readEntities:130
2. 2021-01-27 10:22:04 0.488 GET 200 /api/1.0/info/prepaidAddBalance?ctn=7768741498&hash
Because elastic can't take different timestamps at the same time, I cut milliceconds in first timepattern, but it doesn't work, nothing goes to elastic.
My current filter is:
filter {
if "/uss/servlet/ivrRequest" in [message] {
grok { match =\> {"message" =\> "(?%{YEAR}-%{MONTHNUM}-%{MONTHDAY})\t[\t]%{WORD:method}[\t]%{URIPATH:request\_type}[?]BNumber=%{NUMBER:bnumber}&ANumber=%{NUMBER:msisdn}[\t]%{NUMBER:response\_code}[\t]%{NUMBER:response\_time}"}}
mutate {
add\_field =\> {"log\_time" =\> "%{date} %{hms}"}
}
date { match =\> ["log\_time", "YYYY-MM-dd HH:mm:ss"]
timezone =\> "Asia/Almaty"
target =\> "@timestamp"
}
mutate {strip =\> ["log\_time"]
remove\_field =\> ["date", "hms"]
}
}
else if "/uss/servlet/ussdHttpServlet.ru" in [message] or "/uss/servlet/ussdHttpServlet.kz" in [message] {
grok { match =\> {"message" =\> "(?%{YEAR}-%{MONTHNUM}-%{MONTHDAY})\t[\t]%{WORD:method}[\t]%{URIPATH:request\_type}[?]msisdn=%{NUMBER:msisdn}&ussd=%{NUMBER:ussd}[\t]%{NUMBER:response\_code}[\t]%{NUMBER:response\_time}"}}
mutate {
add\_field =\> {"log\_time" =\> "%{date} %{hms}"}
}
date { match =\> ["log\_time", "YYYY-MM-dd HH:mm:ss"]
timezone =\> "Asia/Almaty"
target =\> "@timestamp"
}
mutate {strip =\> ["log\_time"]
remove\_field =\> ["date", "hms"]
}
}
if "ER000" in [message] or "ER001" in [message] or "ER002" in [message] or "ER003" in [message] or "ER004" in [message] or "ER004" in [message] or "TUX" in [message] or "ER034" in [message] {
grok { match =\> {"message" =\> "[%{TIMESTAMP\_ISO8601:error\_log\_time}]%{GREEDYDATA} %{LOGLEVEL:log-level}%{GREEDYDATA} \NAPIERROR %{WORD:action}%{GREEDYDATA} [%{WORD:response\_code}%{GREEDYDATA}\BAN=[%{NUMBER:ban}%{GREEDYDATA}\SUBSCRIBER\_NO=[%{NUMBER:subscriber\_no}%{GREEDYDATA}\SOC=[%{WORD:soc}%{GREEDYDATA}"}}
mutate { gsub =\> ["error\_log\_time", ",\d{3}$", ""]}
date { match =\> ["error\_log\_time", "YYYY-MM-dd HH:mm:ss"]
timezone =\> "Asia/Almaty"
target =\> "@timestamp"
}
}
else if "! service is too slow" in [message] {
grok { match =\> {"message" =\> "[%{TIMESTAMP\_ISO8601:error\_log\_time}]%{GREEDYDATA} %{LOGLEVEL:log-level}%{GREEDYDATA}- %{GREEDYDATA:response\_code}"}}
mutate { gsub =\> ["error\_log\_time", ",\d{3}$", ""]}
date { match =\> ["error\_log\_time", "YYYY-MM-dd HH:mm:ss"]
timezone =\> "Asia/Almaty"
target =\> "@timestamp"
}
}
else if "! Tuxedo service is too slow" in [message] {
grok { match =\> {"message" =\> "[%{TIMESTAMP\_ISO8601:error\_log\_time}]%{GREEDYDATA} %{LOGLEVEL:log-level}%{GREEDYDATA}- %{GREEDYDATA:response\_code}: com%{GREEDYDATA}"}}
mutate { gsub =\> ["error\_log\_time", ",\d{3}$", ""]}
date { match =\> ["error\_log\_time", "YYYY-MM-dd HH:mm:ss"]
timezone =\> "Asia/Almaty"
target =\> "@timestamp"
}
}
else {
grok { match =\> {"message" =\> "(?%{YEAR}-%{MONTHNUM}-%{MONTHDAY})\t[\t]%{NUMBER:response\_time}[\t]%{WORD:method}[\t]%{NUMBER:response\_code}[\t]%{URIPATH:request\_type}[?]ctn=%{NUMBER:msisdn}"}}
mutate {
add\_field =\> {"log\_time" =\> "%{date} %{hms}"}
}
date { match =\> ["log\_time", "YYYY-MM-dd HH:mm:ss"]
timezone =\> "Asia/Almaty"
target =\> "@timestamp"
}
mutate {strip =\> ["log\_time"]
remove\_field =\> ["date", "hms"]
}
}
if "\_grokparsefailure" in [tags] {
drop {}
}
}
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [March 9, 2021, 4:11pm UTC](https://discuss.elastic.co/t/2-different-time/266640/2 "2021-03-09T16:11:48Z")
You grok patterns cannot be compiled. Please edit your post, select the configuration, and click on \ in the toolbar above the edit panel. That will change the formatting from
grok { match =\> {"message" =\> "[%{TIMESTAMP\_ISO8601:error\_log\_time}]%
to
```
grok { match => {"message" => "[%{TIMESTAMP_ISO8601:error_log_time}]%
```
and prevent characters from the pattern being interpreted as formatting.
---
### Author:  [@111435](https://discuss.elastic.co/u/111435)
#### Post date: [March 10, 2021, 7:48am UTC](https://discuss.elastic.co/t/2-different-time/266640/3 "2021-03-10T07:48:24Z")
```
filter {
if "/uss/servlet/ivrRequest" in [message] {
grok { match => {"message" => "(?%{YEAR}-%{MONTHNUM}-%{MONTHDAY})[\t](?%{HOUR}:%{MINUTE}:%{SECOND})[\t]%{WORD:method}[\t]%{URIPATH:request_type}[\?]BNumber=%{NUMBER:bnumber}\&ANumber=%{NUMBER:msisdn}[\t]%{NUMBER:response_code}[\t]%{NUMBER:response_time}"}}
mutate {
add_field => {"log_time" => "%{date} %{hms}"}
}
date { match => ["log_time", "YYYY-MM-dd HH:mm:ss"]
timezone => "Asia/Almaty"
target => "@timestamp"
}
mutate {strip => ["log_time"]
remove_field => ["date", "hms"]
}
}
else if "/uss/servlet/ussdHttpServlet.ru" in [message] or "/uss/servlet/ussdHttpServlet.kz" in [message] {
grok { match => {"message" => "(?%{YEAR}-%{MONTHNUM}-%{MONTHDAY})[\t](?%{HOUR}:%{MINUTE}:%{SECOND})[\t]%{WORD:method}[\t]%{URIPATH:request_type}[\?]msisdn=%{NUMBER:msisdn}\&ussd=%{NUMBER:ussd}[\t]%{NUMBER:response_code}[\t]%{NUMBER:response_time}"}}
mutate {
add_field => {"log_time" => "%{date} %{hms}"}
}
date { match => ["log_time", "YYYY-MM-dd HH:mm:ss"]
timezone => "Asia/Almaty"
target => "@timestamp"
}
mutate {strip => ["log_time"]
remove_field => ["date", "hms"]
}
}
if "ER000" in [message] or "ER001" in [message] or "ER002" in [message] or "ER003" in [message] or "ER004" in [message] or "ER004" in [message] or "TUX" in [message] or "ER034" in [message] {
grok { match => {"message" => "\[%{TIMESTAMP_ISO8601:error_log_time}\]%{GREEDYDATA} %{LOGLEVEL:log-level}%{GREEDYDATA} \NAPIERROR %{WORD:action}%{GREEDYDATA} \[%{WORD:response_code}%{GREEDYDATA}\BAN=\[%{NUMBER:ban}%{GREEDYDATA}\SUBSCRIBER_NO=\[%{NUMBER:subscriber_no}%{GREEDYDATA}\SOC=\[%{WORD:soc}%{GREEDYDATA}"}}
mutate { gsub => ["error_log_time", "\,\d{3}$", ""]}
date { match => ["error_log_time", "YYYY-MM-dd HH:mm:ss"]
timezone => "Asia/Almaty"
target => "@timestamp"
}
}
else if "! service is too slow" in [message] {
grok { match => {"message" => "\[%{TIMESTAMP_ISO8601:error_log_time}\]%{GREEDYDATA} %{LOGLEVEL:log-level}%{GREEDYDATA}\- %{GREEDYDATA:response_code}"}}
mutate { gsub => ["error_log_time", "\,\d{3}$", ""]}
date { match => ["error_log_time", "YYYY-MM-dd HH:mm:ss"]
timezone => "Asia/Almaty"
target => "@timestamp"
}
}
else if "! Tuxedo service is too slow" in [message] {
grok { match => {"message" => "\[%{TIMESTAMP_ISO8601:error_log_time}\]%{GREEDYDATA} %{LOGLEVEL:log-level}%{GREEDYDATA}\- %{GREEDYDATA:response_code}\: com%{GREEDYDATA}"}}
mutate { gsub => ["error_log_time", "\,\d{3}$", ""]}
date { match => ["error_log_time", "YYYY-MM-dd HH:mm:ss"]
timezone => "Asia/Almaty"
target => "@timestamp"
}
}
else {
grok { match => {"message" => "(?%{YEAR}-%{MONTHNUM}-%{MONTHDAY})[\t](?%{HOUR}:%{MINUTE}:%{SECOND})[\t]%{NUMBER:response_time}[\t]%{WORD:method}[\t]%{NUMBER:response_code}[\t]%{URIPATH:request_type}[\?]ctn=%{NUMBER:msisdn}"}}
mutate {
add_field => {"log_time" => "%{date} %{hms}"}
}
date { match => ["log_time", "YYYY-MM-dd HH:mm:ss"]
timezone => "Asia/Almaty"
target => "@timestamp"
}
mutate {strip => ["log_time"]
remove_field => ["date", "hms"]
}
}
if "_grokparsefailure" in [tags] {
drop {}
}
}
```
---
### Author:  [@111435](https://discuss.elastic.co/u/111435)
#### Post date: [March 10, 2021, 7:55am UTC](https://discuss.elastic.co/t/2-different-time/266640/4 "2021-03-10T07:55:34Z")
Thank you I changed.
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [March 10, 2021, 4:34pm UTC](https://discuss.elastic.co/t/2-different-time/266640/5 "2021-03-10T16:34:33Z")
> [@111435](#):
>
> - [2021-02-24 00:00:06,335] [[ACTIVE] ExecuteThread: '121' for queue: 'weblogic.kernel.Default (self-tuning)'] ERROR EntityDataSourceRepository:readEntities:130
> - 2021-01-27 10:22:04 0.488 GET 200 /api/1.0/info/prepaidAddBalance?ctn=7768741498&hash
For the first, none of these conditionals match
```
if "/uss/servlet/ivrRequest" in [message] {
} else if "/uss/servlet/ussdHttpServlet.ru" in [message] or "/uss/servlet/ussdHttpServlet.kz" in [message] {
if "ER000" in [message] or "ER001" in [message] or "ER002" in [message] or "ER003" in [message] or "ER004" in [message] or "ER004" in [message] or "TUX" in [message] or "ER034" in [message] {
} else if "! service is too slow" in [message] {
} else if "! Tuxedo service is too slow" in [message] {
```
So it goes through the last grok pattern
```
grok { match => {"message" => "(?%{YEAR}-%{MONTHNUM}-%{MONTHDAY})[\t](?%{HOUR}:%{MINUTE}:%{SECOND})[\t]%{NUMBER:response_time}[\t]%{WORD:method}[\t]%{NUMBER:response_code}[\t]%{URIPATH:request_type}[\?]ctn=%{NUMBER:msisdn}"}}
```
which obviously does not match. Since you have
```
if "_grokparsefailure" in [tags] { drop {} }
```
at the end of your filter the event will be dropped.
For the second, I suspect the problem is that your grok pattern requires the fields to be tab separated. If you change `[\t]` to `\s+` then I would expect you to get
```
"method" => "GET",
"log_time" => "2021-01-27 10:22:04",
"response_code" => "200",
"msisdn" => "7768741498",
"request_type" => "/api/1.0/info/prepaidAddBalance",
"response_time" => "0.488"
```
etc.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [April 7, 2021, 4:34pm UTC](https://discuss.elastic.co/t/2-different-time/266640/6 "2021-04-07T16:34:36Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.