# Index data getting deleted automatically after some time,parsing oracle data from logstash to elastic

**URL:** <https://discuss.elastic.co/t/index-data-getting-deleted-automatically-after-some-time-parsing-oracle-data-from-logstash-to-elastic/299979>\
**Category:** Logstash\
**Tags:** ilm-index-lifecycle-management\
**Created:** [March 17, 2022, 5:13pm UTC](https://discuss.elastic.co/t/index-data-getting-deleted-automatically-after-some-time-parsing-oracle-data-from-logstash-to-elastic/299979 "2022-03-17T17:13:49Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![PRASHANT\_MEHTA](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prashant_mehta/32/101764_2.png) [@PRASHANT\_MEHTA](https://discuss.elastic.co/u/PRASHANT_MEHTA)\
**Post date:** [March 17, 2022, 5:13pm UTC](https://discuss.elastic.co/t/index-data-getting-deleted-automatically-after-some-time-parsing-oracle-data-from-logstash-to-elastic/299979/1 "2022-03-17T17:13:50Z")

</div>

Hello All,  
I'm trying to get data from oracle db and parsing it through logstash pipeline and finally to elastic.  
while checking the data in discover in kibana ,observed that the data gets automaticaly deleted after few minutes.what could be causing this?...cross checked pipeline and associated policy.  
In output filter id is unique key in db that might not cause this issue .  
Below are the details,Can someone help out with this?

input{  
jdbc {  
jdbc\_connection\_string =\> "jdbc:oracle:thin:@td08appl002.group.kpti:1991/cis"  
jdbc\_user =\> "MIAADMIN"  
jdbc\_password =\> "MIAADMIN"  
jdbc\_driver\_library =\> "../lib/ojdbc8-12.2.0.1.jar"  
jdbc\_driver\_class =\> "Java::oracle.jdbc.driver.OracleDriver"  
jdbc\_paging\_enabled =\> true  
last\_run\_metadata\_path =\> "../config/lastrun-audit.yml"  
schedule =\> "\*/25 \* \* \* \* \*"  
connection\_retry\_attempts =\> 5  
connection\_retry\_attempts\_wait\_time =\> 10  
statement=\> "select ID, STATUS\_EVENT\_DATA, from\_tz(CAST (CREATION\_DATE AS TIMESTAMP), 'UTC') as CREATION\_DATE from SYSAUDIT where CREATION\_DATE \>= SYS\_EXTRACT\_UTC(:sql\_last\_value )"  
type =\> "audit"  
}

}

output{

elasticsearch {  
hosts =\> "[http://td08appl002.group.kpti:9200](http://td08appl002.group.kpti:9200)"  
ilm\_pattern =\> "{now/d}-000001"  
"doc\_as\_upsert" =\> true   
ilm\_rollover\_alias =\> "job-audit"  
ilm\_policy =\> "audit-policy"  
"document\_id" =\> "%{id}" ---------------unique key in table

```
}

```

## }

## PUT \_ilm/policy/audit-policy { "policy": { "phases": { "hot": { "min\_age": "0ms", "actions": { "rollover": { "max\_age": "1d" }, "set\_priority": { "priority": null } } }, "delete": { "min\_age": "30d", "actions": { "delete": { "delete\_searchable\_snapshot": true } } } } } }

Thanx

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 14, 2022, 5:14pm UTC](https://discuss.elastic.co/t/index-data-getting-deleted-automatically-after-some-time-parsing-oracle-data-from-logstash-to-elastic/299979/2 "2022-04-14T17:14:18Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.