# Twitter input

**URL:** <https://discuss.elastic.co/t/twitter-input/54284>\
**Category:** Logstash\
**Created:** [June 29, 2016, 1:26pm UTC](https://discuss.elastic.co/t/twitter-input/54284 "2016-06-29T13:26:01Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![Diego\_Gomez](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/diego_gomez/32/12893_2.png) [@Diego\_Gomez](https://discuss.elastic.co/u/Diego_Gomez)\
**Post date:** [June 29, 2016, 1:26pm UTC](https://discuss.elastic.co/t/twitter-input/54284/1 "2016-06-29T13:26:01Z")

</div>

this is my code logastash.conf. The problem I have is that there is no index data

```
input {
          twitter { 
         consumer_key => "dplzZiWV2qP5q2dMZpM8lLb0c"
          consumer_secret =>"wpemcks8EqdgY3ZjV8dMJOViB0p QuZsWOPtnH65jHAJdvyyaDW"
     oauth_token => "738404980470730752- lrULPA0EEPZHlMDOyDNVoSFdfvykyKJ"
     oauth_token_secret => "ydtfWx6QZ5a5CTG1R2rA7wCTcBwBVElvOfdVP9Uxo3JE3"
     keywords => ["tecnologia"]
     full_tweet => true
}
}

filter {
  }

 output {
 stdout { codec => dots }
 elasticsearch {
 protocol => ["http"]
 hosts => ["localhost:9200"]
 index => "twitter"
 document_type => "tweet"
  }
}
```

---

<div class="post-metadata">

**Author:** ![Diego\_Gomez](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/diego_gomez/32/12893_2.png) [@Diego\_Gomez](https://discuss.elastic.co/u/Diego_Gomez)\
**Post date:** [June 29, 2016, 1:28pm UTC](https://discuss.elastic.co/t/twitter-input/54284/2 "2016-06-29T13:28:15Z")

</div>

in the fields they appear Kibana me but tell me not index

 ![](https://us1.discourse-cdn.com/elastic/original/2X/a/ac2b67d08f384191a4258152bd2d5ce29adf2754.JPG)

---

<div class="post-metadata">

**Author:** ![Diego\_Gomez](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/diego_gomez/32/12893_2.png) [@Diego\_Gomez](https://discuss.elastic.co/u/Diego_Gomez)\
**Post date:** [June 29, 2016, 1:31pm UTC](https://discuss.elastic.co/t/twitter-input/54284/3 "2016-06-29T13:31:30Z")

</div>

this data gives me logstash

```
     "created_at" => "Tue Jun 28 17:35:37 +0000 2016",
                   "id" => 747845925133967361,
               "id_str" => "747845925133967361",
                 "text" => "El Sol parece adentrarse en una nueva fase evolutiva https://t.co/US9CDau5Na #tecnologia",
                   "source" => "<a href=\"http://twitterfeed.com\" rel=\"nofollow\">twitterfeed</a>",
            "truncated" => false,
"in_reply_to_status_id" => nil,
"in_reply_to_status_id_str" => nil,
  "in_reply_to_user_id" => nil,
  "in_reply_to_user_id_str" => nil,
"in_reply_to_screen_name" => nil,
                 "user" => {
                                "id" => 1678983510,
                            "id_str" => "1678983510",
                              "name" => "Tecnoticias",
                       "screen_name" => "tuitTecnologia",
                          "location" => "Madrid",
                               "url" => nil,
                       "description" => "Padre de @tuitApple y @tuitAndroid. Publicamos cualquier #noticia sobre #ciencia y #tecnología: #móviles, #pcs, #consolas, #internet, #apple, #android y más.",
                         "protected" => false,
                          "verified" => false,
                   "followers_count" => 7588,
                     "friends_count" => 6400,
                      "listed_count" => 202,
                  "favourites_count" => 23,
                    "statuses_count" => 245511,
                        "created_at" => "Sat Aug 17 19:50:14 +0000 2013",
                        "utc_offset" => -14400,
                         "time_zone" => "Eastern Time (US & Canada)",
                       "geo_enabled" => false,
                              "lang" => "es",
              "contributors_enabled" => false,
                     "is_translator" => false,
          "profile_background_color" => "1A1B1F",
      "profile_background_image_url" => "http://abs.twimg.com/images/themes/theme9/bg.gif",
"profile_background_image_url_https" => "https://abs.twimg.com/images/themes/theme9/bg.gif",
           "profile_background_tile" => false,
                "profile_link_color" => "2F39ED",
      "profile_sidebar_border_color" => "FFFFFF",
        "profile_sidebar_fill_color" => "DDEEF6",
                "profile_text_color" => "333333",
      "profile_use_background_image" => true,
                 "profile_image_url" => "http://pbs.twimg.com/profile_images/378800000391393974/5806f3428f11d4cfc6fd4de4ad28b9c3_normal.jpeg",
           "profile_image_url_https" => "https://pbs.twimg.com/profile_images/378800000391393974/5806f3428f11d4cfc6fd4de4ad28b9c3_normal.jpeg",
                "profile_banner_url" => "https://pbs.twimg.com/profile_banners/1678983510/1406578558",
                   "default_profile" => false,
             "default_profile_image" => false,
                         "following" => nil,
               "follow_request_sent" => nil,
                     "notifications" => nil
     },
                  "geo" => nil,
          "coordinates" => nil,
                "place" => nil,
         "contributors" => nil,
      "is_quote_status" => false,
        "retweet_count" => 0,
       "favorite_count" => 0,
             "entities" => {
     "hashtags" => [
    [0] {
           "text" => "tecnologia",
        "indices" => [
            [0] 77,
            [1] 88
        ]
    }
],
         "urls" => [
    [0] {
                 "url" => "https://t.co/US9CDau5Na",
        "expanded_url" => "http://tecnologia.tusueldo.com/14Ol",
         "display_url" => "tecnologia.tusueldo.com/14Ol",
             "indices" => [
            [0] 53,
            [1] 76
        ]
    }
],
"user_mentions" => [],
      "symbols" => []
 },
            "favorited" => false,
            "retweeted" => false,
   "possibly_sensitive" => false,
         "filter_level" => "low",
                 "lang" => "es",
         "timestamp_ms" => "1467135337514",
             "@version" => "1",
           "@timestamp" => "2016-06-28T17:35:37.000Z"
```

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [June 29, 2016, 3:10pm UTC](https://discuss.elastic.co/t/twitter-input/54284/4 "2016-06-29T15:10:31Z")

</div>

> [@Diego\_Gomez](#):
>
> protocol =\> ["http"]

What version of Logstash are you using? `protocol` is not a supported option on recent versions. See [Elasticsearch output plugin | Logstash Reference [8.11] | Elastic](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html)

---

<div class="post-metadata">

**Author:** ![Diego\_Gomez](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/diego_gomez/32/12893_2.png) [@Diego\_Gomez](https://discuss.elastic.co/u/Diego_Gomez)\
**Post date:** [June 29, 2016, 3:34pm UTC](https://discuss.elastic.co/t/twitter-input/54284/5 "2016-06-29T15:34:45Z")

</div>

hi Aaron  
I'm using version 2.3.1

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [June 29, 2016, 3:49pm UTC](https://discuss.elastic.co/t/twitter-input/54284/6 "2016-06-29T15:49:38Z")

</div>

The configuration as you've given should not even be able to run. Here below, I tested a simple example with similar configuration parameters (including `protocol`):

```auto
input {
  stdin {}
}

filter {}

output {
  elasticsearch {
    hosts => ["127.0.0.1:9200"]
    index => "dummy"
    document_type => "foo"
    protocol => ["http"]
  }
}

```

And when I try to run it, this is the output:

```auto
$ bin/logstash -f ~/logstash-test.conf --verbose
starting agent {:level=>:info}
starting pipeline {:id=>"main", :level=>:info}
Settings: Default pipeline workers: 8
Unknown setting 'protocol' for elasticsearch {:level=>:error}
Pipeline aborted due to error {:exception=>#<LogStash::ConfigurationError: Something is wrong with your configuration.>, :backtrace=>["/opt/logstash-2.3.2/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.2-java/lib/logstash/config/mixin.rb:134:in `config_init'", "/opt/logstash-2.3.2/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.2-java/lib/logstash/outputs/base.rb:63:in `initialize'", "/opt/logstash-2.3.2/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.2-java/lib/logstash/output_delegator.rb:74:in `register'", "/opt/logstash-2.3.2/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.2-java/lib/logstash/pipeline.rb:181:in `start_workers'", "org/jruby/RubyArray.java:1613:in `each'", "/opt/logstash-2.3.2/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.2-java/lib/logstash/pipeline.rb:181:in `start_workers'", "/opt/logstash-2.3.2/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.2-java/lib/logstash/pipeline.rb:136:in `run'", "/opt/logstash-2.3.2/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.2-java/lib/logstash/agent.rb:465:in `start_pipeline'"], :level=>:error}
stopping pipeline {:id=>"main"}
Closing inputs {:level=>:info}
Closed inputs {:level=>:info}

```

Please note the error message: `Unknown setting 'protocol' for elasticsearch {:level=>:error}`

---

<div class="post-metadata">

**Author:** ![Diego\_Gomez](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/diego_gomez/32/12893_2.png) [@Diego\_Gomez](https://discuss.elastic.co/u/Diego_Gomez)\
**Post date:** [June 29, 2016, 4:15pm UTC](https://discuss.elastic.co/t/twitter-input/54284/7 "2016-06-29T16:15:06Z")

</div>

I do not take the http protocol and provides us with elasticsearch\_java plugin, and see the same Kibana

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [June 29, 2016, 4:32pm UTC](https://discuss.elastic.co/t/twitter-input/54284/8 "2016-06-29T16:32:17Z")

</div>

I _highly_ recommend you switch to using the standard elasticsearch output plugin soon, as there is a strong possibility that the node protocol will be removed soon. The transport protocol may also go away in future versions.

That said, if you're having Kibana configuration issues, you should take those questions to [https://discuss.elastic.co/c/kibana](https://discuss.elastic.co/c/kibana)

---

<div class="post-metadata">

**Author:** ![Diego\_Gomez](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/diego_gomez/32/12893_2.png) [@Diego\_Gomez](https://discuss.elastic.co/u/Diego_Gomez)\
**Post date:** [June 29, 2016, 4:46pm UTC](https://discuss.elastic.co/t/twitter-input/54284/9 "2016-06-29T16:46:50Z")

</div>

> [@Diego\_Gomez](#):
>
> "geo" =\> nil

the fields are empty as " geo" = \> nil in logstash may be because it takes a long time to take that data, and for that reason logstash passes empty , they said I could use REST API but not how to use it. You know if you can use?

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [June 29, 2016, 5:16pm UTC](https://discuss.elastic.co/t/twitter-input/54284/10 "2016-06-29T17:16:13Z")

</div>

Are all of the geo fields empty, or just some? I mean, it seems possible that some tweets just don't have that information because of how they were submitted, or from where (no reverse lookup of the location possible).

---

<div class="post-metadata">

**Author:** ![Diego\_Gomez](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/diego_gomez/32/12893_2.png) [@Diego\_Gomez](https://discuss.elastic.co/u/Diego_Gomez)\
**Post date:** [June 29, 2016, 8:27pm UTC](https://discuss.elastic.co/t/twitter-input/54284/11 "2016-06-29T20:27:14Z")

</div>

no data in any field geo

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [June 29, 2016, 9:04pm UTC](https://discuss.elastic.co/t/twitter-input/54284/12 "2016-06-29T21:04:58Z")

</div>

You should run with `--debug` turned on and see what comes up.

It may be that Twitter isn't sending the geo field. It may be that something else is going on.

---

<div class="post-metadata">

**Author:** ![Diego\_Gomez](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/diego_gomez/32/12893_2.png) [@Diego\_Gomez](https://discuss.elastic.co/u/Diego_Gomez)\
**Post date:** [June 30, 2016, 12:49pm UTC](https://discuss.elastic.co/t/twitter-input/54284/13 "2016-06-30T12:49:37Z")

</div>

> [@theuntergeek](#):
>
> You should run with --debug turned on and see what comes up.
> 
> It may be that Twitter isn't sending the geo field. It may be that something else is going on.

ok thank you

---

<div class="post-metadata">

**Author:** ![Diego\_Gomez](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/diego_gomez/32/12893_2.png) [@Diego\_Gomez](https://discuss.elastic.co/u/Diego_Gomez)\
**Post date:** [June 30, 2016, 12:58pm UTC](https://discuss.elastic.co/t/twitter-input/54284/14 "2016-06-30T12:58:48Z")

</div>

It shows me this error and is not referred

```
 Twitter too many requests error, sleeping for s {:level=>:warn, :file=>"logstash/inputs/twitter.rb", :line=>"134", :method=>"run"}
```

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [July 1, 2016, 7:52pm UTC](https://discuss.elastic.co/t/twitter-input/54284/15 "2016-07-01T19:52:47Z")

</div>

Well, that suggests why you aren't seeing data: Twitter is throttling you.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:49am UTC](https://discuss.elastic.co/t/twitter-input/54284/16 "2017-07-06T04:49:50Z")

</div>