Hi all,
I´ve a elapsed filter but the calculation of the elapsed time is based on the timestamp value the occurs when the data enters the Elastic not the value i'm getting from the DB.
I've seen a couple of posts saying to override the timestamp field but i can't make it work.
Here is my configuration:
input {
jdbc {
jdbc_driver_library => "D:\elk\logstash-6.1.1\vendor\ibm\db2jcc.jar"
jdbc_driver_class => "com.ibm.db2.jcc.DB2Driver"
jdbc_connection_string => "XXXXX"
jdbc_user => "XXXX"
jdbc_password => "XXXXX"
tracking_column => date_created
tracking_column_type => "timestamp"
last_run_metadata_path => "D:\elk\logstash-6.1.1\metadata\logstash_jdbc_qld_v2_last_run"
use_column_value => true
schedule => "* * * * *"
statement => "SELECT DATE_CREATED,
DESTINATION_HOST, OPERATION_ID, MESSAGE
FROM COMMUNICATION_LOG
WHERE DATE_CREATED > :sql_last_value "
}
}
filter {
date {
match => [ "date_created", "yyyy-MM-dd HH:mm:ss,SSS"]
target => "@timestamp"
}
if [message_type] == "Req" {
mutate {
add_tag => [ "taskStarted" ]
}
}
if [message_type] == "Res" {
mutate {
add_tag => [ "taskTerminated" ]
}
}
elapsed {
unique_id_field => "operation_id"
start_tag => "taskStarted"
end_tag => "taskTerminated"
timeout => 30
}
}
output {
elasticsearch {
hosts => ["127.0.0.1:9200"]
index => "eai_qua_log_v2-%{+YYYY.MM.dd}"
document_id =>"%{communication_id}"
}
}
I'm still getting the diference:
- "@timestamp": "2019-02-01T16:32:00.156Z"
- "date_created": "2019-02-01T16:30:56.738Z",
And the elapsed time is calculated with the timestamp, so the values are not real.
Can anyone help?
Cheers.