# Need to configure message timestamp as timestamp in elastic stack
**URL:** https://discuss.elastic.co/t/need-to-configure-message-timestamp-as-timestamp-in-elastic-stack/288191
**Category:** Logstash
**Created:** [November 2, 2021, 6:06am UTC](https://discuss.elastic.co/t/need-to-configure-message-timestamp-as-timestamp-in-elastic-stack/288191 "2021-11-02T06:06:50Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@Mobin](https://discuss.elastic.co/u/Mobin)
#### Post date: [November 2, 2021, 6:06am UTC](https://discuss.elastic.co/t/need-to-configure-message-timestamp-as-timestamp-in-elastic-stack/288191/1 "2021-11-02T06:06:50Z")
We have created an elastic stack to monitor logs from the Linux clients. On Linux clients, filebeat is configured to ship the logs to the elastic stack.
When we are searching for the logs using elastic discover option @timestamp and message timestamp is not matching. How can we configure timestamp as same as message timestamp?
eg: fields ( time, @timestamp, beat.hostname, ,message, source)
November 1st 2021, 08:41:20.986 November 1st 2021, 08:41:20.986 scnmgmt3 "Nov 1 06:00:03 scnmgmt3 sshd[84934]: Accepted password for root from 172.22.76.114 port 43778 ssh2"
/var/log/secure
logstash input:-
```auto
input {
beats {
port => 5044
ssl => false
}
}
```
logstash filter:-
```auto
filter
{
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGLINE}" }
}
date {
match => ["timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"]
}
}
}
```
logstash output:-
```auto
output {
elasticsearch {
hosts => ["localhost:9200"]
sniffing => true
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}
```
filebeat config:-
```auto
filebeat.prospectors:
- input_type: log
paths:
- /var/log/messages
- /var/log/secure
output.logstash:
hosts: ["172.x.x.x:5044"]
```
---
### Author:  [@Mustafa\_NAJIB](https://discuss.elastic.co/u/Mustafa_NAJIB)
#### Post date: [November 2, 2021, 11:38am UTC](https://discuss.elastic.co/t/need-to-configure-message-timestamp-as-timestamp-in-elastic-stack/288191/2 "2021-11-02T11:38:52Z")
Hello :
this is your input :
```auto
November 1st 2021, 08:41:20.986 scnmgmt3 "Nov 1 06:00:03 scnmgmt3 sshd[84934]: Accepted password for root from 172.22.76.114 port 43778 ssh2"/var/log/secure
```
this is the new filter :
```auto
grok {
match => {
"message" => [
"^%{GREEDYDATA:time} %{DATA:beat_hostname} \"%{GREEDYDATA:TIMESTAMP} %{DATA:beat_hostname} %{GREEDYDATA}: %{GREEDYDATA:log_message}\"%{GREEDYDATA:source}",
"%{GREEDYDATA:FAILPARSE}"
]
}
overwrite => ["beat_hostname"]
}
```
OUTPUT :
```auto
{
"TIMESTAMP": "Nov 1 06:00:03",
"beat_hostname": "scnmgmt3",
"log_message": "Accepted password for root from 172.22.76.114 port 43778 ssh2",
"message": "November 1st 2021, 08:41:20.986 scnmgmt3 \"Nov 1 06:00:03 scnmgmt3 sshd[84934]: Accepted password for root from 172.22.76.114 port 43778 ssh2\"/var/log/secure",
"source": "/var/log/secure",
"time": "November 1st 2021, 08:41:20.986"
}
```
---
### Author:  [@Mobin](https://discuss.elastic.co/u/Mobin)
#### Post date: [November 3, 2021, 6:07am UTC](https://discuss.elastic.co/t/need-to-configure-message-timestamp-as-timestamp-in-elastic-stack/288191/3 "2021-11-03T06:07:48Z")
Thanks for the reply.
I have changed the filter
```auto
filter
{
if [type] == "syslog" {
grok {
match => {
"message" => [
"^%{GREEDYDATA:time} %{DATA:beat_hostname} \"%{GREEDYDATA:TIMESTAMP} %{DATA:beat_hostname} %{GREEDYDATA}: %{GREEDYDATA:log_message}\"%{GREEDYDATA:source}",
}
}
'''
"%{GREEDYDATA:FAILPARSE}"
]
}
overwrite => ["beat_hostname"]
}
```
Still I am not able to search time from message filed. In the kibana there is no available fileds for @time.
---
### Author:  [@Rios](https://discuss.elastic.co/u/Rios)
#### Post date: [November 3, 2021, 6:51am UTC](https://discuss.elastic.co/t/need-to-configure-message-timestamp-as-timestamp-in-elastic-stack/288191/4 "2021-11-03T06:51:06Z")
Avoid to use GREEDYDATA,it's expensive.
Make a custom pattern:
`%{MONTH} %{MONTHDAY}.{0,2} %{YEAR}, %{TIME}\.%{NONNEGINT}`
Then you have to convert to date "November 1st 2021 08:41:20.986" with
```auto
date {
match => ["timestamp", "MMM d yyyy HH:mm:ss.SSS", "MMM d yyyy, HH:mm:ss.SSS"]
```
At the end, only correct format the time field must be like: _"time": "2021-11-01T08:41:20.986Z"_
---
### Author:  [@Mobin](https://discuss.elastic.co/u/Mobin)
#### Post date: [November 3, 2021, 8:03am UTC](https://discuss.elastic.co/t/need-to-configure-message-timestamp-as-timestamp-in-elastic-stack/288191/5 "2021-11-03T08:03:52Z")
My requirement is to grep the events based on time. I am already able to do that but the problem is the timestamp in the message filed is not matching with filebeat timestamp. So I need to add a new filed as same as message timestamp and the same needs to be added to filebeat fields.
---
### Author:  [@Rios](https://discuss.elastic.co/u/Rios)
#### Post date: [November 3, 2021, 11:59am UTC](https://discuss.elastic.co/t/need-to-configure-message-timestamp-as-timestamp-in-elastic-stack/288191/6 "2021-11-03T11:59:22Z")
@timestamp - time from Logstash, default datetime format
timestamp - time from Filebeat, default string format
If you need to @timestamp became as timestamp, just copy. But the "timestamp" field must be in date format converted not string.
```auto
date {
match => ["timestamp", "MMM d yyyy HH:mm:ss.SSS", "MMM d yyyy, HH:mm:ss.SSS"]
}
```
Then copy after conversion
```auto
mutate {
copy => { "timestamp" => "@timestamp"}
}
```
---
### Author:  [@Mobin](https://discuss.elastic.co/u/Mobin)
#### Post date: [November 5, 2021, 5:06am UTC](https://discuss.elastic.co/t/need-to-configure-message-timestamp-as-timestamp-in-elastic-stack/288191/7 "2021-11-05T05:06:26Z")
Dear Rios,
Sorry. Since I am new to elk, I didn't understand your solution.
We are using file beat to ship the logs from the client and elk server logstash input created to receive the logs from filebeat. I didn't understand why we need to change the timestamp here.
In our elk server there is a timestamp ( I think that is based on filebeat sync time ) and one more is there on the filebeat message field ( It is a single field wiith time stamp and message ). Now we need to seperate the timestamp from the message field So we can query based on the exact time of the event/message.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 3, 2021, 5:07am UTC](https://discuss.elastic.co/t/need-to-configure-message-timestamp-as-timestamp-in-elastic-stack/288191/8 "2021-12-03T05:07:22Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.