Logstash input: Twitter

[Rysiu]

Hello,

I have a simple pipeline setup in Logstash (keys changed of course):

input {
        twitter {
                consumer_key => "rcLfXIhprrPKLgVI467Slbux0"
                consumer_secret =>"uYk68znfoDQq70lfWXS3IxscHOffQWzZK5nkpbeE3D9ue9JDOd"
                oauth_token => "1292007035622632472-91pHS6gKnwfCAKKVuKughJb2L12lDF"
                oauth_token_secret => "BSeK1qyEEGtocP4omiqu7KxPj4Ie8ufpG6zTdZS3BxFnH"
                keywords => ["union"]
                languages=> ["en"]
                full_tweet => true
                ignore_retweets => true
        }
}


output {
        elasticsearch {
                hosts => ["https://node000.elk.int:9200"]
                cacert => '/etc/logstash/certs/ca.crt'
                user => 'userrrr'
                password => 'passssssss'

                manage_template => false
                index => "twitter"
        }
}

Unfortunately Logstash returns an error:

[2022-02-27T09:38:28,378][WARN ][logstash.inputs.twitter  ][twitter][96a78dc08f18a92c07b1f152323c5e9d5b90690fbc8d9873cf01d05bf7d7c373] Twitter client error {:message=>"", :exception=>Twitter::Error::Forbidden, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/twitter-6.2.0/lib/twitter/streaming/response.rb:24:in `on_headers_complete'", "org/ruby_http_parser/RubyHttpParser.java:370:in `<<'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/twitter-6.2.0/lib/twitter/streaming/response.rb:19:in `<<'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/twitter-6.2.0/lib/twitter/streaming/connection.rb:20:in `stream'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/twitter-6.2.0/lib/twitter/streaming/client.rb:119:in `request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/twitter-6.2.0/lib/twitter/streaming/client.rb:38:in `filter'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-twitter-4.0.3/lib/logstash/inputs/twitter.rb:151:in `do_run'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-twitter-4.0.3/lib/logstash/inputs/twitter.rb:131:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:405:in `inputworker'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:396:in `block in start_input'"], :options=>nil}

The Forbidden error indicates some problem with the endpoint (?).
Information for Forbidden error? Twitter API Response Codes & Error Support | Twitter Developer Platform

Probably the problem is the "Callback URI / Redirect URL" item in the Twitter API configuration (webiste: developer.twitter.com).

The Callback URI / Redirect URL field is required.

Where could the problem be?

[leandrojmp (Leandro Pereira)]

I'm not sure this input still works, the Twitter API changed and the gem used by this input was not updated to be compatible with the new version.

If your Access Key is for the Twitter API V2, then this is the issue, check this github issue about it.

Unless the gem is updated or a new input is built, you won't be able to use the twitter input with the new twitter API.