Logstash delete all documents before http_poller

Elastic Stack Logstash
1

Hi,

How can I remove old data of an index before http_polling ?

I call Jira API every day with http_poller. Every call fetches data from Jira and creates documents. My problem is that I have finally too many duplicated documents in my index.

What is the best way to remove all documents from the index, before calling http_poller ?

2

Hi Dogee, Welcome to the Elastic community.

Could you please share your pipeline configuration ? Also are you certain you getting unique data on every http polling call ?

3

Hi,
I finally find a solution to my problem, by using the field issue_key as @documentId. Each document is then updated rather than duplicated after subsequent calls.
Thanks.