Date issue

Log file date is on 26th but in elasticsearch date show as 25th why ?

Logstash conf file.

input {
beats {
port => 5044
}
}
filter {
grok {
match => {
"message" => "%{TIMESTAMP_ISO8601:timestamp}%{GREEDYDATA:lt_message}"}
}
kv {
source => "lt_message"
include_keys => ["Transaction", "Duration", "Txn_Status", "Project_Name", "KBytes_sec", "Parameters"]
trim_value => " {}\r\n"
}
date {
match => ["timestamp", "yyyy-MM-dd HH:mm:ss.SSS"]
target => "@timestamp"
}
mutate {
convert => {"Duration" => "float"}
remove_field => ["@version", "timestamp","source","beat","input","host","tags","prospector","offset","message", "lt_message", "log", "agent", "fields", "ecs", "rest" ]
remove_tag => ["tags"]
}
}
output {
file {
path => "/elastic/log/logstash-lt.log"
}
elasticsearch {
hosts => ["http://lthppc-wdc-lg13:9200/"]
index => "loadtest"
}
}

Log file

2021-04-26 05:01:40.613 Transaction = "SYM_PublicEval_LT_S01_T01_Product1" Duration = 0.881012 Txn_Status=PASS Project_Name=LT_VPP_Comm_WCM_EPP KBytes_sec=1258 Parameters=

2021-04-26 05:01:40.627 Transaction = "SYM_PublicEval_LT_S01_T01_Product1" Duration = 0.881065 Txn_Status=PASS Project_Name=LT_VPP_Comm_WCM_EPP KBytes_sec=1258 Parameters=

2021-04-26 05:01:40.608 Transaction = "SYM_PublicEval_LT_S01_T01_Product1" Duration = 0.915010 Txn_Status=PASS Project_Name=LT_VPP_Comm_WCM_EPP KBytes_sec=1212 Parameters=

2021-04-26 05:01:40.631 Transaction = "SYM_PublicEval_LT_S01_T01_Product1" Duration = 0.951052 Txn_Status=PASS Project_Name=LT_VPP_Comm_WCM_EPP KBytes_sec=1166 Parameters=

2021-04-26 05:01:40.657 Transaction = "SYM_PublicEval_LT_S01_T01_Product1" Duration = 0.946489 Txn_Status=PASS Project_Name=LT_VPP_Comm_WCM_EPP KBytes_sec=1171 Parameters=

2021-04-26 05:01:40.769 Transaction = "SYM_PublicEval_LT_S01_T01_Product1" Duration = 0.897121 Txn_Status=PASS Project_Name=LT_VPP_Comm_WCM_EPP KBytes_sec=1236 Parameters=

2021-04-26 05:01:40.881 Transaction = "SYM_PublicEval_LT_S01_T01_Product1" Duration = 0.828352 Txn_Status=PASS Project_Name=LT_VPP_Comm_WCM_EPP KBytes_sec=1338 Parameters=

2021-04-26 05:01:40.849 Transaction = "SYM_PublicEval_LT_S01_T01_Product1" Duration = 0.887204 Txn_Status=PASS Project_Name=LT_VPP_Comm_WCM_EPP KBytes_sec=1249 Parameters=

2021-04-26 05:01:40.734 Transaction = "SYM_PublicEval_LT_S01_T01_Product1" Duration = 1.011191 Txn_Status=PASS Project_Name=LT_VPP_Comm_WCM_EPP KBytes_sec=1096 Parameters=

2021-04-26 05:01:40.832 Transaction = "SYM_PublicEval_LT_S01_T01_Product1" Duration = 0.932400 Txn_Status=PASS Project_Name=LT_VPP_Comm_WCM_EPP KBytes_sec=1189 Parameters=

2021-04-26 05:01:40.848 Transaction = "SYM_PublicEval_LT_S01_T01_Product1" Duration = 0.928241 Txn_Status=PASS Project_Name=LT_VPP_Comm_WCM_EPP KBytes_sec=1194 Parameters=

2021-04-26 05:01:40.879 Transaction = "SYM_PublicEval_LT_S01_T01_Product1" Duration = 0.903992 Txn_Status=PASS Project_Name=LT_VPP_Comm_WCM_EPP KBytes_sec=1226 Parameters=

Sharing screen shot.

logfile & kibana time stamp are in PST .

In elasticsearch i see data as 26 but in kibana showing as 25th
{
"_index" : "loadtest",
"_type" : "_doc",
"_id" : "SXvvDXkB71udp0fEj7a2",
"_score" : null,
"_source" : {
"Txn_Status" : "PASS",
"Transaction" : "SYM_PublicEval_LT_S01_T01_Product1",
"@timestamp" : "2021-04-26T04:27:44.170Z",
"Parameters" : """
""",
"KBytes_sec" : "488",
"Duration" : 2.272314,
"Project_Name" : "LT_VPP_Comm_WCM_EPP"
},
"sort" : [
1619411264170
]
},
{
"_index" : "loadtest",
"_type" : "_doc",
"_id" : "S3vvDXkB71udp0fEj7a2",
"_score" : null,
"_source" : {
"Txn_Status" : "PASS",
"Transaction" : "SYM_PublicEval_LT_S01_T01_Product1",
"@timestamp" : "2021-04-26T04:27:44.208Z",
"Parameters" : """
""",
"KBytes_sec" : "493",
"Duration" : 2.24742,
"Project_Name" : "LT_VPP_Comm_WCM_EPP"
},

Hi @sasvmware ,

I would try

And don't remove tags before this is working. The tags can be quite useful when debugging issues.

Hope that helps :slight_smile:

In logstash and elasticsearch timestamps are always in UTC, kibana will adjust that to the local timezone of the browser. That is 8 hours behind PST, 7 behind PDT, so times early on the 26th in PST will be late on the 25th in UTC.