Twitter Sentiment Analysis

Elastic Stack Elasticsearch
1

Hi All,

How can we do twitter sentiment analysis in Kibana.

I used Config file to load data from twitter to elastic search.With this i am able to load data in ES & Kibana.

In Config file i provided some keywords.Its searching for those keywords and giving me data.
In this mostly i am getting irrelevant data ,Is their any way to get data from particular hashtag to get relevant data what we can use for sentiment analysis. How we can enhance this data by adding sentiment analysis and how we can use this data to make decisions.

after getting data confused in kibana side how can i show sentiment analysis.How to filter good and bad.
How can i filter words with good bucket & bad bucket data
Is their any other application i need to install do sentiment analysis.

Please tag some links if you know any examples.
Currently i am working on windows environment.

Thanks & Regards,
Dan.

2

Sentiment would be a metadata field like any other field that Kibana can summarise (language, country, author....).
The values for sentiment would obviously be positive, negative, neutral etc
This is not hard for Kibana/Elasticsearch to handle.

The big question is how you prepare the data ie how you determine if a tweet is positive, negative or neutral in your data preparation pipeline. This is not easy, especially if you are considering different languages, cultures, age groups etc. It's a notoriously hard thing to do accurately. Companies like Datasift provide services that attempt to do this data enrichment on tweets. I found another one here [1] but I'm sure there's many more. They are all typically using models trained for particular types of content and may not apply well to the content you are studying - yesterday's #fail may be #FML tomorrow.

[1] http://blog.datumbox.com/how-to-build-your-own-twitter-sentiment-analysis-tool/

3

Hi Mark Harwood,

i am thinking like whenever we create configure file their only we can filter words with good,bad and average.

suppose "Love" is a good word we filter this word in logstash config file only.That means will get data for good. Like this will do the same thing for remaining words.Here we need a count and we don't need data.

But i am stucked at how to filter data by text.I used below filter but it's not working.

filter {
grok {
match => ["message", "mingus text[%{WORD:love}]: %{GREEDYDATA}"]
}
}

output {
elasticsearch{
if [messagetype] == "INFO" {
stdout {
codec => "rubydebug"
}
}
protocol => "http"
document_id => "%{id}"
host => "localhost"
index => "twitter"
index_type => "tweet"
}
}

Please help me.

Regards,
Dinesh

4

I'm not the best person to ask about Logstash I'm afraid. This question is about preparing content upstream of elasticsearch so this perhaps is not the best forum.
You can try the Logstash forum here: https://discuss.elastic.co/c/logstash although I expect a quick google for "twitter sentiment analysis" will provide more specialised software for this task.