# Twitter and Logstash config files

**URL:** <https://discuss.elastic.co/t/twitter-and-logstash-config-files/53655>\
**Category:** Logstash\
**Created:** [June 22, 2016, 12:21pm UTC](https://discuss.elastic.co/t/twitter-and-logstash-config-files/53655 "2016-06-22T12:21:50Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![jamesl](https://avatars.discourse-cdn.com/v4/letter/j/c0e974/32.png) [@jamesl](https://discuss.elastic.co/u/jamesl)\
**Post date:** [June 22, 2016, 12:21pm UTC](https://discuss.elastic.co/t/twitter-and-logstash-config-files/53655/1 "2016-06-22T12:21:50Z")

</div>

Hi. Trying to get Twitter feeds into my ES System but its not working like the way it should?  
If I remove the 'full\_tweet = true' from the input file I get some of the message (see attached) but not any Geo or other goodness...  
Below is my Logstash config and attached is an example tweet that I currently get in ES.

Also, when I put full\_tweet back in my /var/log/logstash/logstash.log file fills up with lots of 'unauthorized' messages and just scrolls with errors.

root@log01:/etc/logstash/conf.d# cat \*  
input {  
twitter {  
consumer\_key =\> "A"  
consumer\_secret =\> "B"  
oauth\_token =\> "C"  
oauth\_token\_secret =\> "D"  
keywords =\> ["I","am","testing","this"]  
type =\> "twitter-log01"  
}  
beats {  
port =\> 5044  
type =\> "beats-log01"  
}  
beats {  
port =\> 5045  
type =\> "winfilebeat-log01"  
}  
tcp {  
port =\> 5514  
type =\> "syslog5514-log01"  
}  
tcp {  
port =\> 5515  
type =\> "syslog5515-log01"  
}  
}  
filter {  
if [type] == "syslog5514-log01" {  
grok {  
match =\> { "message" =\> "%{SYSLOGTIMESTAMP:syslog\_timestamp} %{SYSLOGHOST:syslog\_hostname} %{DATA:syslog\_program}(?:[%{POSINT:syslog\_pid}])?: %{GREEDYDATA:syslog\_message}" }  
add\_field =\> ["received\_at", "%{@timestamp}"]  
add\_field =\> ["received\_from", "%{host}"]  
}  
syslog\_pri { }  
date {  
match =\> ["syslog\_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"]  
}  
}  
}  
output {  
if [type] == "twitter-log01" {  
elasticsearch {  
hosts =\> ["els03","els04"]  
sniffing =\> true  
manage\_template =\> false  
index =\> "twitter-%{+YYYY.MM.dd}"  
document\_type =\> "twitter-log01"  
}  
} else if [type] == "beats-log01" {  
elasticsearch {  
hosts =\> ["els03","els04"]  
sniffing =\> true  
manage\_template =\> false  
index =\> "%{[@metadata][beat]}-%{+YYYY.MM.dd}"  
document\_type =\> "%{[@metadata][type]}"  
}  
} else if [type] == "winfilebeat-log01" {  
elasticsearch {  
hosts =\> ["els03","els04"]  
sniffing =\> true  
manage\_template =\> false  
index =\> "winfilebeat-%{+YYYY.MM.dd}"  
document\_type =\> "winfilebeat-log01"  
}  
} else if [type] == "syslog5514-log01" {  
elasticsearch {  
hosts =\> ["els03","els04"]  
sniffing =\> true  
manage\_template =\> false  
index =\> "syslog5514-%{+YYYY.MM.dd}"  
document\_type =\> "syslog5514-log01"  
}  
} else if [type] == "syslog5515-log01" {  
elasticsearch {  
hosts =\> ["els03","els04"]  
sniffing =\> true  
manage\_template =\> false  
index =\> "syslog5515-%{+YYYY.MM.dd}"  
document\_type =\> "syslog5515-log01"  
}  
} else {  
elasticsearch {  
hosts =\> ["els03","els04"]  
sniffing =\> true  
manage\_template =\> false  
index =\> "%{[@metadata][beat]}-%{+YYYY.MM.dd}"  
document\_type =\> "%{[@metadata][type]}"  
}  
}  
}

 ![](https://us1.discourse-cdn.com/elastic/original/2X/4/483dcc150d89d5472910a1006364bc8aa6bf57b6.png)

Thanks!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:51am UTC](https://discuss.elastic.co/t/twitter-and-logstash-config-files/53655/2 "2017-07-06T04:51:13Z")

</div>