# Can I replace the @timestamp of the dashboard with the time at which the event get logged in the file
**URL:** https://discuss.elastic.co/t/can-i-replace-the-timestamp-of-the-dashboard-with-the-time-at-which-the-event-get-logged-in-the-file/355607
**Category:** Logstash
**Created:** [March 18, 2024, 1:11pm UTC](https://discuss.elastic.co/t/can-i-replace-the-timestamp-of-the-dashboard-with-the-time-at-which-the-event-get-logged-in-the-file/355607 "2024-03-18T13:11:55Z")
**Posts on this page:** 18
**Page:** 1
### Author:  [@Shrimad\_Mishra](https://discuss.elastic.co/u/Shrimad_Mishra)
#### Post date: [March 18, 2024, 1:11pm UTC](https://discuss.elastic.co/t/can-i-replace-the-timestamp-of-the-dashboard-with-the-time-at-which-the-event-get-logged-in-the-file/355607/1 "2024-03-18T13:11:55Z")
Hi there,
I wanted to replace the @timestamp value with the log time. I am using filebeat as a logs collector.
Here is my filter value which I am trying
```auto
grok {
match => { "message" => "\[%{DATA:datetime}\] INFO \[.*\] %{GREEDYDATA:logger_json}" }
}
date {
match => ["DATA:timestamp", "dd-MMM-yyyy HH:mm:ss"]
target => "@timestamp"
}
json {
source => "logger_json"
target => "logger_json_parsed"
}
```
And here is the sample of the log messages
[13-Mar-2024 07:42:38] INFO [:10] [GenAIVoiceBot] Server 1 app log number 99707
[13-Mar-2024 07:42:38] INFO [:10] [GenAIVoiceBot] Server 1 app log number 99708
[13-Mar-2024 07:42:38] INFO [:10] [GenAIVoiceBot] Server 1 app log number 99709
[13-Mar-2024 07:42:38] INFO [:10] [GenAIVoiceBot] Server 1 app log number 99710
[13-Mar-2024 07:42:38] INFO [:10] [GenAIVoiceBot] Server 1 app log number 99711
[13-Mar-2024 07:42:38] INFO [:10] [GenAIVoiceBot] Server 1 app log number 99712
[13-Mar-2024 07:42:38] INFO [:10] [GenAIVoiceBot] Server 1 app log number 99713
[13-Mar-2024 07:42:38] INFO [:10] [GenAIVoiceBot] Server 1 app log number 99714
[13-Mar-2024 07:42:38] INFO [:10] [GenAIVoiceBot] Server 1 app log number 99715
[13-Mar-2024 07:42:38] INFO [:10] [GenAIVoiceBot] Server 1 app log number 99716
[13-Mar-2024 07:42:38] INFO [:10] [GenAIVoiceBot] Server 1 app log number 99717
[13-Mar-2024 07:42:38] INFO [:10] [GenAIVoiceBot] Server 1 app log number 99718
[13-Mar-2024 07:42:38] INFO [:10] [GenAIVoiceBot] Server 1 app log number 99719
[13-Mar-2024 07:42:38] INFO [:10] [GenAIVoiceBot] Server 1 app log number 99720
[13-Mar-2024 07:42:38] INFO [:10] [GenAIVoiceBot] Server 1 app log number 99721
Please help
---
### Author:  [@Rios](https://discuss.elastic.co/u/Rios)
#### Post date: [March 18, 2024, 3:05pm UTC](https://discuss.elastic.co/t/can-i-replace-the-timestamp-of-the-dashboard-with-the-time-at-which-the-event-get-logged-in-the-file/355607/2 "2024-03-18T15:05:13Z")
You should use the datetime field, not timestamp.
```auto
date {
match => ["datetime", "dd-MMM-yyyy HH:mm:ss"]
target => "@timestamp"
}
```
---
### Author:  [@Shrimad\_Mishra](https://discuss.elastic.co/u/Shrimad_Mishra)
#### Post date: [March 18, 2024, 4:01pm UTC](https://discuss.elastic.co/t/can-i-replace-the-timestamp-of-the-dashboard-with-the-time-at-which-the-event-get-logged-in-the-file/355607/3 "2024-03-18T16:01:38Z")
I do not why but after replacing DATA:timestamp with datetime, the logs are not getting pushed to OpenSearch
It started pushing logs by using the below config
```auto
filter {
if [fields][log_type] == "monitor" {
mutate {
add_field => { "log_location" => "call_monitoring_log" }
}
grok {
match => { "message" => "\[%{DATA:datetime}\] INFO \[.*\] %{GREEDYDATA:logger_json}" }
}
date {
match => ["DATA:datetime", "dd-MMM-yyyy HH:mm:ss"]
target => "@timestamp"
}
json {
source => "logger_json"
target => "logger_json_parsed"
}
}
}
```
but still the timestamp and the datetime is not same
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 18, 2024, 4:01pm UTC](https://discuss.elastic.co/t/can-i-replace-the-timestamp-of-the-dashboard-with-the-time-at-which-the-event-get-logged-in-the-file/355607/4 "2024-03-18T16:01:38Z")
OpenSearch/OpenDistro are AWS run products and differ from the original Elasticsearch and Kibana products that Elastic builds and maintains. You may need to contact them directly for further assistance.
(This is an automated response from your friendly Elastic bot. Please report this post if you have any suggestions or concerns :elasticheart: )
---
### Author:  [@Rios](https://discuss.elastic.co/u/Rios)
#### Post date: [March 18, 2024, 10:39pm UTC](https://discuss.elastic.co/t/can-i-replace-the-timestamp-of-the-dashboard-with-the-time-at-which-the-event-get-logged-in-the-file/355607/5 "2024-03-18T22:39:16Z")
Can you show JSON record from Kibana before changing? What is in "tags"
Have you tested your new .conf after date changes?
The logger\_json field doesn't contain any JSON structure, it's plain text.
---
### Author:  [@Shrimad\_Mishra](https://discuss.elastic.co/u/Shrimad_Mishra)
#### Post date: [March 19, 2024, 4:21am UTC](https://discuss.elastic.co/t/can-i-replace-the-timestamp-of-the-dashboard-with-the-time-at-which-the-event-get-logged-in-the-file/355607/6 "2024-03-19T04:21:31Z")
```auto
{
"message" => "[18-Mar-2024 23:30:41] INFO [:49] [GenAIVoiceBot] 998 {\"@timestamp\": \"Mar 18, 2024 @ 23:30:41.954588\"}",
"@version" => "1",
"ecs" => {
"version" => "8.0.0"
},
"fields" => {
"log_type" => "monitor"
},
"log_location" => "call_monitoring_log",
"logger_json_parsed" => {
"@timestamp" => "Mar 18, 2024 @ 23:30:41.954588"
},
"@timestamp" => 2024-03-18T18:00:42.233Z,
"container" => {
"id" => "call_monitoring_logger.log"
},
"logger_json" => "998 {\"@timestamp\": \"Mar 18, 2024 @ 23:30:41.954588\"}",
"tags" => [
[0] "beats_input_codec_plain_applied"
],
"log" => {
"offset" => 552379,
"file" => {
"path" => "...",
"inode" => "6195882"
}
},
"cloud" => {
"provider" => "openstack",
"input" => {
"type" => "filestream"
},
"datetime" => "18-Mar-2024 23:30:41"
}
```
here is the sysout log, I am using logstash for my usecase
---
### Author:  [@Rios](https://discuss.elastic.co/u/Rios)
#### Post date: [March 19, 2024, 6:47am UTC](https://discuss.elastic.co/t/can-i-replace-the-timestamp-of-the-dashboard-with-the-time-at-which-the-event-get-logged-in-the-file/355607/7 "2024-03-19T06:47:57Z")
And which the @timestamp field should be used: at the begging ` [18-Mar-2024 23:30:41]` or at the end: `@timestamp: "Mar 18, 2024 @ 23:30:41.954588"` ?
---
### Author:  [@Shrimad\_Mishra](https://discuss.elastic.co/u/Shrimad_Mishra)
#### Post date: [March 19, 2024, 6:56am UTC](https://discuss.elastic.co/t/can-i-replace-the-timestamp-of-the-dashboard-with-the-time-at-which-the-event-get-logged-in-the-file/355607/8 "2024-03-19T06:56:25Z")
The first one should be used
---
### Author:  [@Rios](https://discuss.elastic.co/u/Rios)
#### Post date: [March 19, 2024, 9:20am UTC](https://discuss.elastic.co/t/can-i-replace-the-timestamp-of-the-dashboard-with-the-time-at-which-the-event-get-logged-in-the-file/355607/9 "2024-03-19T09:20:06Z")
You have wrongly added DATA in match, here: ` match => ["DATA:datetime"`
I have changed dates to be more visible.
```auto
input {
generator {
"message" => '[11-Mar-2024 21:10:11] INFO [:49] [GenAIVoiceBot] 998 {"@timestamp": "Mar 18, 2024 @ 23:30:41.954588"}'
count => 1 }
}
filter {
#if [fields][log_type] == "monitor" { # temporarily commented
mutate { add_field => { "log_location" => "call_monitoring_log" } }
grok { match => { "message" => "\[%{DATA:datetime}\] INFO \[.*\] %{GREEDYDATA:logger_json}" } }
date {
match => ["datetime", "dd-MMM-yyyy HH:mm:ss"]
target => "@timestamp"
}
json {
source => "logger_json"
target => "logger_json_parsed"
}
#}
}
output {
stdout {codec => rubydebug{} }
}
```
Result:
```auto
{
"@timestamp" => 2024-03-11T20:10:11.000Z,
"message" => "[11-Mar-2024 21:10:11] INFO [:49] [GenAIVoiceBot] 998 {\"@timestamp\": \"Mar 18, 2024 @ 23:30:41.954588\"}",
"datetime" => "11-Mar-2024 21:10:11",
"logger_json" => "998 {\"@timestamp\": \"Mar 18, 2024 @ 23:30:41.954588\"}",
"logger_json_parsed" => {
"@timestamp" => "Mar 18, 2024 @ 23:30:41.954588"
},
"log_location" => "call_monitoring_log"
}
```
- "@timestamp" will get the value from the first field and will be as the date type.
- datetime will be as string field. If you don't need you can replace with [@metadata][datetime] or simply remove\_field.
- [logger\_json\_parsed][@timestamp] is another field as string.
---
### Author:  [@Shrimad\_Mishra](https://discuss.elastic.co/u/Shrimad_Mishra)
#### Post date: [March 19, 2024, 9:35am UTC](https://discuss.elastic.co/t/can-i-replace-the-timestamp-of-the-dashboard-with-the-time-at-which-the-event-get-logged-in-the-file/355607/10 "2024-03-19T09:35:51Z")
Okay so what is the finally config, I am confused a bit now,
Tried the same which you told but now the logs are not getting pushed to OpenSearch
Here is the error which I am getting
Error parsing json {:source=\>"logger\_json", :raw=\>"{'@timestamp': '2024-03-19T15:22:43.911625'}", :exception=\>#\
### Author:  [@Shrimad\_Mishra](https://discuss.elastic.co/u/Shrimad_Mishra)
#### Post date: [March 19, 2024, 1:35pm UTC](https://discuss.elastic.co/t/can-i-replace-the-timestamp-of-the-dashboard-with-the-time-at-which-the-event-get-logged-in-the-file/355607/11 "2024-03-19T13:35:26Z")
If I want the last timestamp to be there instead of first what would be the conf?
---
### Author:  [@Rios](https://discuss.elastic.co/u/Rios)
#### Post date: [March 19, 2024, 1:55pm UTC](https://discuss.elastic.co/t/can-i-replace-the-timestamp-of-the-dashboard-with-the-time-at-which-the-event-get-logged-in-the-file/355607/12 "2024-03-19T13:55:09Z")
You should add after json:
```auto
date {
match => ["[logger_json_parsed][@timestamp]", "MMM dd, yyyy @ HH:mm:ss.SSSSSS"]
target => "@timestamp" # or any other name
}
```
---
### Author:  [@Shrimad\_Mishra](https://discuss.elastic.co/u/Shrimad_Mishra)
#### Post date: [March 19, 2024, 2:32pm UTC](https://discuss.elastic.co/t/can-i-replace-the-timestamp-of-the-dashboard-with-the-time-at-which-the-event-get-logged-in-the-file/355607/13 "2024-03-19T14:32:58Z")
No it is not working
---
### Author:  [@Shrimad\_Mishra](https://discuss.elastic.co/u/Shrimad_Mishra)
#### Post date: [March 19, 2024, 3:51pm UTC](https://discuss.elastic.co/t/can-i-replace-the-timestamp-of-the-dashboard-with-the-time-at-which-the-event-get-logged-in-the-file/355607/14 "2024-03-19T15:51:35Z")
Also In the above answer which you have given for datetime, I changed the datetime to UTC in the logger and due to this the logs are getting pushed to OpenSearch so is there any way to convert @timstam to UTC after mapping so that I do not need to refractor my code much.
---
### Author:  [@Shrimad\_Mishra](https://discuss.elastic.co/u/Shrimad_Mishra)
#### Post date: [March 19, 2024, 5:05pm UTC](https://discuss.elastic.co/t/can-i-replace-the-timestamp-of-the-dashboard-with-the-time-at-which-the-event-get-logged-in-the-file/355607/15 "2024-03-19T17:05:55Z")
@Rios please help this is bit urgent, help me with fixing the below command as per the need
```auto
sudo docker run --restart unless-stopped --log-driver local --log-opt max-size=10m --memory="1g" --cpus="1.0" --oom-kill-disable -dit -p port:port -v mount:/var/log/logstash --name test --user=root opensearchproject/logstash-oss-with-opensearch-output-plugin:7.16.2 -e '
input {
beats {
port => 5045
}
}
filter {
if [fields][log_type] == "monitor" {
mutate {
add_field => { "log_location" => "call_monitoring_log" }
}
grok {
match => { "message" => "\[%{DATA:datetime}\] INFO \[.*\] %{GREEDYDATA:logger_json}" }
}
date {
match => ["datetime", "dd-MMM-yyyy HH:mm:ss"]
target => "@timestamp"
}
json {
source => "logger_json"
target => "logger_json_parsed"
}
}
}
output {
if [log_location] == "call_monitoring_log" {
opensearch {
}
stdout {
codec => rubydebug
}
}
}'
```
[19-Mar-2024 22:19:18] INFO [:31] [GenAIVoiceBot] {'@timestamp': 'Mar-19-2024 @ 22:19:18.689375'}
[19-Mar-2024 22:25:12] INFO [:31] [GenAIVoiceBot] {'@timestamp': 'Mar-19-2024 @ 22:25:12.960077'}
[19-Mar-2024 22:28:57] INFO [:31] [GenAIVoiceBot] {'@timestamp': 'Mar-19-2024 @ 22:28:57.907195'}
[19-Mar-2024 22:29:40] INFO [:31] [GenAIVoiceBot] {'@timestamp': 'Mar-19-2024 @ 22:29:40.122095'}
is the log format
After replace @timestamp with the datetime is should change to UTC time and push to OpenSearch
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [March 19, 2024, 5:27pm UTC](https://discuss.elastic.co/t/can-i-replace-the-timestamp-of-the-dashboard-with-the-time-at-which-the-event-get-logged-in-the-file/355607/16 "2024-03-19T17:27:44Z")
> [@Shrimad\_Mishra](#):
>
> Error parsing json {:source=\>"logger\_json", :raw=\>"{'@timestamp': '2024-03-19T15:22:43.911625'}", :exception=\>#\ ["logger_json", "'", '"'] }
```
---
### Author:  [@Rios](https://discuss.elastic.co/u/Rios)
#### Post date: [March 20, 2024, 2:12am UTC](https://discuss.elastic.co/t/can-i-replace-the-timestamp-of-the-dashboard-with-the-time-at-which-the-event-get-logged-in-the-file/355607/17 "2024-03-20T02:12:23Z")
For the last case:
```auto
input {
generator {
#"message" => '[11-Mar-2024 21:10:11] INFO [:49] [GenAIVoiceBot] 998 {"@timestamp": "Mar 12, 2024 @ 23:30:31.954588"}'
"message" => "[19-Mar-2024 22:25:12] INFO [:31] [GenAIVoiceBot] {'@timestamp': 'Mar-19-2024 @ 22:25:12.960077'}"
count => 1 }
}
filter {
#if [fields][log_type] == "monitor" {
mutate { add_field => { "log_location" => "call_monitoring_log" } }
grok { match => { "message" => "\[%{DATA:datetime}\] INFO \[.*\] %{GREEDYDATA:logger_json}" } }
# date {
# match => ["datetime", "dd-MMM-yyyy HH:mm:ss"]
# target => "@timestamp"
# }
mutate { gsub => ["logger_json", "'", '"'] }
json {
source => "logger_json"
target => "logger_json_parsed"
}
date {
match => ["[logger_json_parsed][@timestamp]", "MMM-dd-yyyy @ HH:mm:ss.SSSSSS", "MMM dd, yyyy @ HH:mm:ss.SSSSSS"]
target => "@timestamp"
}
#}
}
output {
stdout {codec => rubydebug{} }
}
```
Result:
```auto
{
"datetime" => "19-Mar-2024 22:25:12",
"logger_json_parsed" => {
"@timestamp" => "Mar-19-2024 @ 22:25:12.960077"
},
"message" => "[19-Mar-2024 22:25:12] INFO [:31] [GenAIVoiceBot] {'@timestamp': 'Mar-19-2024 @ 22:25:12.960077'}",
"log_location" => "call_monitoring_log",
"logger_json" => "{\"@timestamp\": \"Mar-19-2024 @ 22:25:12.960077\"}",
"@timestamp" => 2024-03-19T21:25:12.960Z
}
```
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [April 17, 2024, 2:13am UTC](https://discuss.elastic.co/t/can-i-replace-the-timestamp-of-the-dashboard-with-the-time-at-which-the-event-get-logged-in-the-file/355607/18 "2024-04-17T02:13:08Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.