# How to delete index and schedule data in logstash config file

**URL:** <https://discuss.elastic.co/t/how-to-delete-index-and-schedule-data-in-logstash-config-file/153844>\
**Category:** Logstash\
**Created:** [October 24, 2018, 3:55pm UTC](https://discuss.elastic.co/t/how-to-delete-index-and-schedule-data-in-logstash-config-file/153844 "2018-10-24T15:55:20Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Nethaji](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nethaji/32/72803_2.png) [@Nethaji](https://discuss.elastic.co/u/Nethaji)\
**Post date:** [October 24, 2018, 3:55pm UTC](https://discuss.elastic.co/t/how-to-delete-index-and-schedule-data-in-logstash-config-file/153844/1 "2018-10-24T15:55:20Z")

</div>

Hi I have scheduled my data in logstash config file it's working fine but the duplicate data will come in index.

so i tried to delete the index first and then update the data but it's not working that is config file is running but the index is not deleted again duplicate will come

please provide solution for how to delete the index first and update data later

I have attached my config file

input {  
exec{  
type =\> "curl"  
command =\> 'curl -XDELETE "[http://ipaddress:9200/schedule](http://ipaddress:9200/schedule)"'  
codec =\> json  
interval =\> 120  
}  
jdbc {  
jdbc\_connection\_string =\> "jdbc:mysql://192.168.**:3306/dbname"  
jdbc\_user =\> " ****"  
jdbc\_password =\> "**\*_"  
jdbc\_validate\_connection =\> true  
jdbc\_paging\_enabled =\> "true"  
jdbc\_page\_size =\> 100000  
jdbc\_driver\_library =\> "D:\ELK\com.mysql.jdbc\_5.1.5.jar"  
jdbc\_driver\_class =\> "com.mysql.jdbc.Driver"  
# our query  
clean\_run =\> true  
schedule =\> "_ \* \* \* \*"  
statement =\> "SELECT \* FROM tbl\_schedule"  
}  
}  
output {   
elasticsearch { hosts =\> ["[http://ipaddress:9200](http://ipaddress:9200)"]  
index =\> "schedule"  
}  
stdout { codec =\> rubydebug }   
}

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [October 24, 2018, 9:27pm UTC](https://discuss.elastic.co/t/how-to-delete-index-and-schedule-data-in-logstash-config-file/153844/2 "2018-10-24T21:27:27Z")

</div>

Why do you want to delete the data?

---

<div class="post-metadata">

**Author:** ![Nethaji](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nethaji/32/72803_2.png) [@Nethaji](https://discuss.elastic.co/u/Nethaji)\
**Post date:** [October 25, 2018, 5:53am UTC](https://discuss.elastic.co/t/how-to-delete-index-and-schedule-data-in-logstash-config-file/153844/3 "2018-10-25T05:53:48Z")

</div>

Because of duplicate will come in my index

Eg: First time I upload 10 data, after schedule it takes another 10 data it continues for every scheduling, so I want to delete the index first and upload.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [October 25, 2018, 5:58am UTC](https://discuss.elastic.co/t/how-to-delete-index-and-schedule-data-in-logstash-config-file/153844/4 "2018-10-25T05:58:47Z")

</div>

Why not just use a custom `_id` for each document, so that the next time it gets sent to Elasticsearch it just overwrites the existing one?

---

<div class="post-metadata">

**Author:** ![Nethaji](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nethaji/32/72803_2.png) [@Nethaji](https://discuss.elastic.co/u/Nethaji)\
**Post date:** [October 25, 2018, 6:12am UTC](https://discuss.elastic.co/t/how-to-delete-index-and-schedule-data-in-logstash-config-file/153844/5 "2018-10-25T06:12:40Z")

</div>

I tried to add document id in logstash config file it just fetch only one data

---

<div class="post-metadata">

**Author:** ![Nethaji](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nethaji/32/72803_2.png) [@Nethaji](https://discuss.elastic.co/u/Nethaji)\
**Post date:** [October 25, 2018, 6:55am UTC](https://discuss.elastic.co/t/how-to-delete-index-and-schedule-data-in-logstash-config-file/153844/6 "2018-10-25T06:55:17Z")

</div>

input {

jdbc {  
jdbc\_connection\_string =\> "jdbc:mysql://ipaddress/sam\_sit3"  
# The user we wish to execute our statement as  
jdbc\_user =\> "_**"  
jdbc\_password =\> "**__"  
jdbc\_validate\_connection =\> true  
jdbc\_paging\_enabled =\> "true"  
jdbc\_page\_size =\> 100000  
# The path to our downloaded jdbc driver  
jdbc\_driver\_library =\> "D:\ELK\com.mysql.jdbc\_5.1.5.jar"  
jdbc\_driver\_class =\> "com.mysql.jdbc.Driver"  
# our query  
clean\_run =\> true  
schedule =\> "_ \* \* \* \*"

```
statement => "SELECT * FROM tbl_schedule"
}

```

}  
output {

```
    elasticsearch { hosts => ["http://ipaddress:9200"] 
	document_id => "%{s.id}"
	 index => "schedule"
	}
    stdout { codec => rubydebug }

```

}

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 22, 2018, 6:55am UTC](https://discuss.elastic.co/t/how-to-delete-index-and-schedule-data-in-logstash-config-file/153844/7 "2018-11-22T06:55:18Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.