Hello,
I am collecting data from twitter,
I have found something online and I was keep on trying to get twitter data into elastic search
finally I have found we need to use the templete.
I have used this templete
{ "index_patterns": "twitter", "settings": { "number_of_shards": 1, "number_of_replicas": 0, "mapping": { "total_fields": { "limit": "10000" } } }, "mappings": { "tweets": { "properties": { "@timestamp": { "type": "date", "format": "dateOptionalTime" }, "text": { "type": "text" }, "user": { "type": "object", "properties": { "description": { "type": "text" } } }, "coordinates": { "type": "object", "properties": { "coordinates": { "type": "geo_point" } } }, "entities": { "type": "object", "properties": { "hashtags": { "type": "object", "properties": { "text": { "type": "text", "fielddata": true } } } } }, "retweeted_status": { "type": "object", "properties": { "text": { "type": "text" } } } }, "dynamic_templates": [ { "string_template": { "match": "*", "match_mapping_type": "string", "mapping": { "type": "keyword" } } } ] } } }
and this is my logstash configuration file
input { twitter { consumer_key => "e3IiLKMJ7BbFb6cqjvkDgVMRw" consumer_secret => "C0w4IziMLZgI4nS0BrGxnjgQGNkPMnGcxEhtRecll94hIhGoXY" oauth_token => "3610593134-ncC8uCwMezcREpOjngkVS0j8kkD0XqFQNAe5ZFk" oauth_token_secret => "R8RCGZaQTcBIy7ioKKJ2Mr8hVBgFM0bsOUlMxR1soDmXj" keywords => [ "trump", "india", "IAS", "pulwama" ] full_tweet => true ignore_retweets => true } } output { stdout { codec => rubydebug } elasticsearch { hosts => "10.10.0.11:9200" index => "twitterbeat-%{+YYYY.MM.dd}" document_type => "doc" template => "/etc/logstash/conf.d/twitter_template2.json" template_name => "twitter" template_overwrite => true } }
They are not getting segregating into fields
and there is another template file I found online I'll attach it below.
can some help me or share me a template for Twitter and get the geo coordinates