I am using jdbc connection for fetching data from database.
I have single index. Let say Contacts.
I am able to fetch data from database. But what i want is when i run logstash for second time it should only add data to elasticsearch which are new in database.
- Running logstash for first time
ES will have 5 data in 'Contacts' index. Database has 5 data
- Running logstash for second time
ES will have 11 data in 'Contacts' index. Database has 6 data
So what is happening is i am having multiple entries of same data. I want only newly added data when running logstash second tme.
How to achieve this ?