DLQ with start_timestamp is not working correctly


(Shivanshu Bagga) #1

Hi,

Following is my configuration for DLQ input plugin-

 dead_letter_queue {
    path => "/opt/shivanshu/setup/logstash-6.0.0/data/dead_letter_queue"
    start_timestamp => "2026-02-08T06:10:21"
    #commit_offsets => false
    pipeline_id => "main"
    tags => "dlq"
 }

I have some events stored in DLQ and wanna process events only those are added recently. So I use start_timestamp , but even after apply this I'm getting all events . So for testing purpose I changed its value to 2026-02-08T06:10:21, but even in this case I'm getting all events .

Here is my one sample event-

{"Locale":"Desiredresponselocalization215950","Message":{},"EventType":"Usage215940","ResourceName":"/api/startSessionForPerson##","Time":"2018-02-08T01:10:21.595-0500","UserAction":"userActionData1518070221594","SubjectType":"YYTU","ResponsePayload":"responsePayloadData1518070221594","@version":1,"host":"127.0.0.1","UserAgent":"userAgentData1518070221594","ClientId":695,"logger_name":"DemoApplication","CorrelationId":"4861cb44-0921-4da7-21594082ec-e86dcc05be35215940","SubjectId":"59bc52da6fb36c95771797f1215950","ElapsedTime(ms)":1964106054,"SessionId":"DB8BA6C6F897136A533696E84D681AB4215950","ModelId":"modelIdData1518070221594","RequestPayload":"requestPayloadData1518070221594","Status":"MOVED_PERMANENTLY","AdminId":"08340215940","ParentSpanID":"e86dcc05be35-0921-5431cb44-82ec-4da7215950","level":"INFO","Lifecycle":"qw","count":4,"Host":"aaaaaaa.hewitt.com","ResourceVersion":5.5,"Severity":"severityData1518070221594","RoleCd":"roleCdData1518070221594","SrcIP":"10.100.19.23","tags":["dlq"],"@timestamp":"2018-02-08T06:10:21.638Z","ConsumerRefID":"ddbdb1bc-b826-4265-9163-3e81e3712e64215940","SystemInstanceId":"5431cb44-8654-4da7-82ec-e86dcc05be35215950","port":42742,"thread_name":"pool-1-thread-2","level_value":20000,"ConsumerId":"Portal_44","ResourceInstance":"42272_CL1_TBAWS_QC_M1215950","@metdata":{"ip_address":"127.0.0.1"},"externalServiceName":"externalServiceNameData1518070221594","TestCfg":"JJ#UY","RequestType":"POST","SpanID":"5431cb44-0921-4da7-82ec-e86dcc05be35215950","EffectiveDate":"02-08-2018"}

Any luck?

Issue reference link


(system) #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.