# How to use twitter plugin in Logstash 5.x

**URL:** <https://discuss.elastic.co/t/how-to-use-twitter-plugin-in-logstash-5-x/70176>\
**Category:** Logstash\
**Created:** [December 29, 2016, 3:15am UTC](https://discuss.elastic.co/t/how-to-use-twitter-plugin-in-logstash-5-x/70176 "2016-12-29T03:15:25Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![RainTown](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raintown/32/140206_2.png) [@RainTown](https://discuss.elastic.co/u/RainTown)\
**Post date:** [December 29, 2016, 3:15am UTC](https://discuss.elastic.co/t/how-to-use-twitter-plugin-in-logstash-5-x/70176/1 "2016-12-29T03:15:25Z")

</div>

Hi

I want to pull tweets matching particular keywords into ELK, adding a field to show what was matched. I can do this, but solution does not scale and I think there is likely a better way.

Here's my conf files, I think intent is self explanatory.

```
input {  
    twitter {
        # add your data
        consumer_key => "xx"
        consumer_secret => "xx"
        oauth_token => "xx-xx"
        oauth_token_secret => "xx"
        keywords => ["london"]
        full_tweet => true
        add_field => { "myplace" => "london" }
    }
}
input {  
    twitter {
        # add your data
        consumer_key => "xx"
        consumer_secret => "xx"
        oauth_token => "xx-xx"
        oauth_token_secret => "xx"
        keywords => ["paris"]
        full_tweet => true
        add_field => { "myplace" => "paris" }
    }
}
output {  
    elasticsearch {
       index => "twitter-%{+YYYY.MM.dd}"
    }
}

```

I was using if condition on output to output into different indices, but that has same problem.

Once I get over 2 or so input sections, my logs fill up with

[WARN][logstash.inputs.twitter] Twitter too many requests error, sleeping for 300s

which is a message I understand but

1. I don't see how I can control/stop this from my end

and

1. I don't see a better way than the above.

Thanks for suggestions,  
RT

---

<div class="post-metadata">

**Author:** ![RainTown](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raintown/32/140206_2.png) [@RainTown](https://discuss.elastic.co/u/RainTown)\
**Post date:** [January 15, 2017, 11:57pm UTC](https://discuss.elastic.co/t/how-to-use-twitter-plugin-in-logstash-5-x/70176/2 "2017-01-15T23:57:46Z")

</div>

Hi

er, anyone have any ideas on this?

RT

---

<div class="post-metadata">

**Author:** ![aruiztinoco](https://avatars.discourse-cdn.com/v4/letter/a/f1d935/32.png) [@aruiztinoco](https://discuss.elastic.co/u/aruiztinoco)\
**Post date:** [January 30, 2017, 5:38pm UTC](https://discuss.elastic.co/t/how-to-use-twitter-plugin-in-logstash-5-x/70176/3 "2017-01-30T17:38:23Z")

</div>

Maybe you need to register 2 different Twitter applications, one for each input.

---

<div class="post-metadata">

**Author:** ![RainTown](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raintown/32/140206_2.png) [@RainTown](https://discuss.elastic.co/u/RainTown)\
**Post date:** [January 31, 2017, 8:22pm UTC](https://discuss.elastic.co/t/how-to-use-twitter-plugin-in-logstash-5-x/70176/4 "2017-01-31T20:22:15Z")

</div>

I tried that. But API limits are per account, and not per app.

---

<div class="post-metadata">

**Author:** ![aruiztinoco](https://avatars.discourse-cdn.com/v4/letter/a/f1d935/32.png) [@aruiztinoco](https://discuss.elastic.co/u/aruiztinoco)\
**Post date:** [February 1, 2017, 3:44am UTC](https://discuss.elastic.co/t/how-to-use-twitter-plugin-in-logstash-5-x/70176/5 "2017-02-01T03:44:40Z")

</div>

Sorry for not being of any help.

---

<div class="post-metadata">

**Author:** ![RainTown](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raintown/32/140206_2.png) [@RainTown](https://discuss.elastic.co/u/RainTown)\
**Post date:** [February 5, 2017, 9:50pm UTC](https://discuss.elastic.co/t/how-to-use-twitter-plugin-in-logstash-5-x/70176/6 "2017-02-05T21:50:22Z")

</div>

Thank you for trying!!

I have updated to 5.2 today but same behavior.

---

<div class="post-metadata">

**Author:** ![aruiztinoco](https://avatars.discourse-cdn.com/v4/letter/a/f1d935/32.png) [@aruiztinoco](https://discuss.elastic.co/u/aruiztinoco)\
**Post date:** [February 6, 2017, 5:17am UTC](https://discuss.elastic.co/t/how-to-use-twitter-plugin-in-logstash-5-x/70176/7 "2017-02-06T05:17:22Z")

</div>

Hard luck! Not sure, but I am thinking about something like this:

input {  
twitter {  
# add your data  
consumer\_key =\> "xx"  
consumer\_secret =\> "xx"  
oauth\_token =\> "xx-xx"  
oauth\_token\_secret =\> "xx"  
keywords =\> ["london","paris"]  
full\_tweet =\> true  
}  
}  
filter {  
mutate {add\_field =\> { "myplace" =\> "london"}  
}  
if ["paris"] in ["text"] {  
update =\> { "myplace" =\> "paris" }  
}  
}  
output {  
stdout {  
codec =\> dots  
}  
elasticsearch {  
hosts =\> "[http://localhost:9200](http://localhost:9200)"  
index =\> "twitter-%{+YYYY.MM.dd}"  
document\_type =\> "tweet"  
template =\> "twitter\_template.json"   
template\_name =\> "twitter-\*"  
template\_overwrite =\> true  
}  
}

Only one input, "london" in "myplace" from the beginning, and update it to "paris" when this keyword is in the "text" field of the tweet. You also need a template.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 6, 2017, 5:17am UTC](https://discuss.elastic.co/t/how-to-use-twitter-plugin-in-logstash-5-x/70176/8 "2017-03-06T05:17:27Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.