Logstash Twitter plugin issues

Hello Guys.
Yesterday i tried to use Twitter Logstash input plugin to connect to Twitter Stream API to get tweets contain word "Twitter".
the Question is, I run the Logstash for 2 hours and it didn't even return one stream of data and i'm sure it is running correctly and new tweets from twitter contain word "Twitter" are coming.

I decide to run Logstash in debug mode and put its output on STDOUT using:

/usr/share/logstash/bin/logstash --path.settings /etc/logstash/ --path.config /etc/logstash/conf.d/tweets.conf

It run and I waited for data for more than 30 minutes, no data. myself by checking twitter there are new tweets contain word "Twitter".

i'm sure my twitter authentication keys are working because i have tested them with Twurl (curl-like twitter client).

So i'm asking myself where's the problem, my side(Logstash twitter plugin) or it's on twitter stream API side?
Logstash is not giving me an error but it is not even showing me result. how can i get to trace the problem?

Thanks.

Here's my Logstash configuration

input {
  twitter {
    consumer_key => "XXX"
    consumer_secret => "XXX"
    oauth_token => "XXX"
    oauth_token_secret => "XXX"
    keywords => ["twitter"]
    ignore_retweets => true
    full_tweet => true
  }
}

filter {
}

output{
  stdout { codec => json }
}

Hello.
For me the working config is with ommiting keyword and using follows.
Eg.

input {
twitter {
consumer_key => "XX"
consumer_secret => "XX"
oauth_token => "XX"
oauth_token_secret => "XX"
full_tweet => true
ignore_retweets => false
follows => ["XXXXXXX","XXXXXXXXXXXX"]
use_samples => false
languages => ["en", "de"]
}
}

but that will require to specify the array of account user IDs that you follow.

How to debug this more in depth is to increase logging verbosity for the twitter:

curl -XPUT '127.0.0.1:9600/_node/logging?pretty' -H 'Content-Type: application/json' -d'
{
"logger.logstash.inputs.twitter" : "TRACE"
}'

The setting will not persist when you will reload the logstash.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.