Date issue

Log file date is on 26th but in elasticsearch date show as 25th why ?

Logstash conf file.

input {
beats {
port => 5044
}
}
filter {
grok {
match => {
"message" => "%{TIMESTAMP_ISO8601:timestamp}%{GREEDYDATA:lt_message}"}
}
kv {
source => "lt_message"
include_keys => ["Transaction", "Duration", "Txn_Status", "Project_Name", "KBytes_sec", "Parameters"]
trim_value => " {}\r\n"
}
date {
match => ["timestamp", "yyyy-MM-dd HH:mm:ss.SSS"]
target => "@timestamp"
}
mutate {
convert => {"Duration" => "float"}
remove_field => ["@version", "timestamp","source","beat","input","host","tags","prospector","offset","message", "lt_message", "log", "agent", "fields", "ecs", "rest" ]
remove_tag => ["tags"]
}
}
output {
file {
path => "/elastic/log/logstash-lt.log"
}
elasticsearch {
hosts => ["http://lthppc-wdc-lg13:9200/"]
index => "loadtest"
}
}

Log file

2021-04-26 05:01:40.613 Transaction = "SYM_PublicEval_LT_S01_T01_Product1" Duration = 0.881012 Txn_Status=PASS Project_Name=LT_VPP_Comm_WCM_EPP KBytes_sec=1258 Parameters=

2021-04-26 05:01:40.627 Transaction = "SYM_PublicEval_LT_S01_T01_Product1" Duration = 0.881065 Txn_Status=PASS Project_Name=LT_VPP_Comm_WCM_EPP KBytes_sec=1258 Parameters=

2021-04-26 05:01:40.608 Transaction = "SYM_PublicEval_LT_S01_T01_Product1" Duration = 0.915010 Txn_Status=PASS Project_Name=LT_VPP_Comm_WCM_EPP KBytes_sec=1212 Parameters=

2021-04-26 05:01:40.631 Transaction = "SYM_PublicEval_LT_S01_T01_Product1" Duration = 0.951052 Txn_Status=PASS Project_Name=LT_VPP_Comm_WCM_EPP KBytes_sec=1166 Parameters=

2021-04-26 05:01:40.657 Transaction = "SYM_PublicEval_LT_S01_T01_Product1" Duration = 0.946489 Txn_Status=PASS Project_Name=LT_VPP_Comm_WCM_EPP KBytes_sec=1171 Parameters=

2021-04-26 05:01:40.769 Transaction = "SYM_PublicEval_LT_S01_T01_Product1" Duration = 0.897121 Txn_Status=PASS Project_Name=LT_VPP_Comm_WCM_EPP KBytes_sec=1236 Parameters=

2021-04-26 05:01:40.881 Transaction = "SYM_PublicEval_LT_S01_T01_Product1" Duration = 0.828352 Txn_Status=PASS Project_Name=LT_VPP_Comm_WCM_EPP KBytes_sec=1338 Parameters=

2021-04-26 05:01:40.849 Transaction = "SYM_PublicEval_LT_S01_T01_Product1" Duration = 0.887204 Txn_Status=PASS Project_Name=LT_VPP_Comm_WCM_EPP KBytes_sec=1249 Parameters=

2021-04-26 05:01:40.734 Transaction = "SYM_PublicEval_LT_S01_T01_Product1" Duration = 1.011191 Txn_Status=PASS Project_Name=LT_VPP_Comm_WCM_EPP KBytes_sec=1096 Parameters=

2021-04-26 05:01:40.832 Transaction = "SYM_PublicEval_LT_S01_T01_Product1" Duration = 0.932400 Txn_Status=PASS Project_Name=LT_VPP_Comm_WCM_EPP KBytes_sec=1189 Parameters=

2021-04-26 05:01:40.848 Transaction = "SYM_PublicEval_LT_S01_T01_Product1" Duration = 0.928241 Txn_Status=PASS Project_Name=LT_VPP_Comm_WCM_EPP KBytes_sec=1194 Parameters=

2021-04-26 05:01:40.879 Transaction = "SYM_PublicEval_LT_S01_T01_Product1" Duration = 0.903992 Txn_Status=PASS Project_Name=LT_VPP_Comm_WCM_EPP KBytes_sec=1226 Parameters=

Sharing screen shot.

logfile & kibana time stamp are in PST .

In elasticsearch i see data as 26 but in kibana showing as 25th
{
"_index" : "loadtest",
"_type" : "_doc",
"_id" : "SXvvDXkB71udp0fEj7a2",
"_score" : null,
"_source" : {
"Txn_Status" : "PASS",
"Transaction" : "SYM_PublicEval_LT_S01_T01_Product1",
"@timestamp" : "2021-04-26T04:27:44.170Z",
"Parameters" : """
""",
"KBytes_sec" : "488",
"Duration" : 2.272314,
"Project_Name" : "LT_VPP_Comm_WCM_EPP"
},
"sort" : [
1619411264170
]
},
{
"_index" : "loadtest",
"_type" : "_doc",
"_id" : "S3vvDXkB71udp0fEj7a2",
"_score" : null,
"_source" : {
"Txn_Status" : "PASS",
"Transaction" : "SYM_PublicEval_LT_S01_T01_Product1",
"@timestamp" : "2021-04-26T04:27:44.208Z",
"Parameters" : """
""",
"KBytes_sec" : "493",
"Duration" : 2.24742,
"Project_Name" : "LT_VPP_Comm_WCM_EPP"
},

Hi @sasvmware ,

I would try

And don't remove tags before this is working. The tags can be quite useful when debugging issues.

Hope that helps :slight_smile:

In logstash and elasticsearch timestamps are always in UTC, kibana will adjust that to the local timezone of the browser. That is 8 hours behind PST, 7 behind PDT, so times early on the 26th in PST will be late on the 25th in UTC.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.