# Logstash storing JDBC DB results set in some cache and not updating data set from DB

**URL:** <https://discuss.elastic.co/t/logstash-storing-jdbc-db-results-set-in-some-cache-and-not-updating-data-set-from-db/170028>\
**Category:** Logstash\
**Created:** [February 26, 2019, 2:49pm UTC](https://discuss.elastic.co/t/logstash-storing-jdbc-db-results-set-in-some-cache-and-not-updating-data-set-from-db/170028 "2019-02-26T14:49:00Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![ApekshaB](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/apekshab/32/63273_2.png) [@ApekshaB](https://discuss.elastic.co/u/ApekshaB)\
**Post date:** [February 26, 2019, 2:49pm UTC](https://discuss.elastic.co/t/logstash-storing-jdbc-db-results-set-in-some-cache-and-not-updating-data-set-from-db/170028/1 "2019-02-26T14:49:01Z")

</div>

Dear Team,

KIndly find my losgtash conf file:

```auto
input {

    jdbc {
        jdbc_validate_connection => true
        jdbc_connection_string => "jdbc:oracle:thin:@10.1.50.79:1521/ndaie2"
        jdbc_user => "AXIA_SPRINT_DEV"
        jdbc_password => "AXIA_SPRINT_DEV"
	    jdbc_fetch_size => 2000
        #jdbc_paging_enabled => true
        #jdbc_page_size => 20000
        jdbc_driver_library => "D:\Apeksha\logstash-5.4.0\Oracle_JDBC_Driver\ojdbc7.jar"
        jdbc_driver_class => "Java::oracle.jdbc.driver.OracleDriver"
        
		statement => "select id, name, city, to_char(dater, 'DD-MON-YYYY HH12:MI:SS') as dater_time from logstash_try WHERE to_char(dater, 'DD-MON-YYYY HH12:MI:SS') > to_char(CURRENT_DATE - interval '7' day, 'DD-MON-YYYY HH12:MI:SS')
		AND to_char(dater, 'DD-MON-YYYY HH12:MI:SS') > TO_CHAR(TO_DATE(:sql_last_value ,'DD-MON-YYYY HH12:MI:SS'), 'DD-MON-YYYY HH12:MI:SS') ORDER BY dater_time"

				
		use_column_value => true
		tracking_column => "dater_time"
		tracking_column_type => "timestamp"
	#clean_run => true 
		jdbc_paging_enabled => "true"
    	jdbc_page_size => "50000"
		jdbc_default_timezone => "Asia/Kolkata"
    	last_run_metadata_path => "C:\Users\apeksha.bhandari\.logstash_try_001"
		schedule => "*/5 * * * * *"
       }
}

output {
		stdout { codec => json }

		elasticsearch {
		hosts => ["10.1.54.76:9200"]
		index => "india_30"		
		document_id => "%{id}" 
		retry_on_conflict => 3	
		}
    file {
                codec => json_lines
                path => "D:\Apeksha\logstash-5.4.0\india_30.log"
        }
}

```

Issue is that after running query once, logstash stores these entries somewhere in cache, and in the next scheduled query hit, doesnt take the updated rows. Is there any way to clean this cache?  
Same queries when run in oracle give correct output

Regards,  
Apeksha

---

<div class="post-metadata">

**Author:** ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)\
**Post date:** [February 26, 2019, 11:06pm UTC](https://discuss.elastic.co/t/logstash-storing-jdbc-db-results-set-in-some-cache-and-not-updating-data-set-from-db/170028/2 "2019-02-26T23:06:18Z")

</div>

> ## State
> 
> The plugin will persist the `sql_last_value` parameter in the form of a metadata file stored in the configured `last_run_metadata_path` . Upon query execution, this file will be updated with the current value of `sql_last_value` . Next time the pipeline starts up, this value will be updated by reading from the file. If `clean_run` is set to true, this value will be ignored and `sql_last_value` will be set to Jan 1, 1970, or 0 if `use_column_value` is true, as if no query has ever been executed.
> 
> -- [Logtash: JDBC Input Plugin - State](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-jdbc.html#_state)

And:

> ## `last_run_metadata_path`
> 
> - Value type is [string](https://www.elastic.co/guide/en/logstash/current/configuration-file-structure.html#string)
> - Default value is `"$HOME/.logstash_jdbc_last_run"`
> 
> Path to file with last run time
> 
> -- [Logtash: JDBC Input Plugin -`last_run_metadata_path`](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-jdbc.html#plugins-inputs-jdbc-last_run_metadata_path)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 26, 2019, 11:18pm UTC](https://discuss.elastic.co/t/logstash-storing-jdbc-db-results-set-in-some-cache-and-not-updating-data-set-from-db/170028/3 "2019-03-26T23:18:27Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.