I am fetching RSS Feeds using logstash-input-rss plugin. I am mainly facing three problems as below. It will be great if someone can give me insgihts.
- How to avoid repetition of items?
i.e. same item is indexed after the time interval, but I want to index only if a new data is added to the feed URL independent of the time interval.
- There are various feeds which are not compatible, how to fix them and parse them too?
- I'm trying to search using "GET index_name/_search?q=title:*"
Its working fine, I just want to output few fields from the searched query, is it possible?