How to filter tweets of logstash_input_twiiter based on timestapm?

Elastic Stack Logstash
1

here is my twitter input tweets

"_source": {
"created_at": "Wed Aug 10 06:42:48 +0000 2016",
"id": 763264318242783200,
"timestamp_ms": "1470811368891",
"@version": "1",
"@timestamp": "2016-08-10T06:42:48.000Z"
}

and my logstash config file which include twitter input plugin filter and output

input {
twitter {
consumer_key => "lvvoeonCRBOHsLAoTPbion9sK"
consumer_secret => "GNHOFzErJhuo0bNq38JUs7xea2BOktMiLa7tunoGwP0oFKCHrY"
oauth_token => "704578110616936448-gfeSklNrITu7fHIZgjw3nwoZ1S0l0Jl"
oauth_token_secret => "IHiyRJRN09jjdUTGrnesALw4DRle35WyX7pdnI3CtEnJ5"
keywords => [ "afghanistan", "TOLOnews", "kabul", "police"]
full_tweet => true
}
}
filter {
date {
match => ["timestamp" , "MMM d YYY HH:mm:ss", "ISO8601"]
}
}
output {
stdout { codec => dots }
elasticsearch {
hosts => "10.20.1.123"
index => "twitter_news"
document_type => "tweets"
}
}

I want to just get new tweets for example today date is 2016-11-16, then I just want to get tweets that have @timestamp= 2016-11-16 not @timestamp= 2016-11-15 or past days tweets, but with this configuration i get past tweets as well, any one help me to how do this ?

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.