Exessive field mapping Twitter to Elasticsearch

Elastic Stack Logstash
1

Hey guys

I am using the ElasticStack_twitter configuration to ingest tweets. (ES, Logstash and Kibana are on version 5.2)

https://github.com/elastic/examples/tree/master/ElasticStack_twitter

I have applied the exact same config and templates from this commit.

When I view the data in Kibana, it tells me that I have over 900 fields being indexed. This eventually fails with an error "more than 10000 fields in an index".

How would I only index the following related fields for any keyword searched:

created_at
id
text
source
user
geo
coordinates
place
retweeted_status
retweet_count
favorite_count
favorited
retweeted
lang

Is this an elasticsearch mapping change or a Logstash filter change?

As always, thank you!

2

You could try setting full_tweet => false.

3

Hey @warkolm, tried that. Same result I'm afraid.

4

Found the problem.

When I edited the original github supplied ES template, I included a grievous typo.

Please consider this post closed and solved.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.