Missing records from mysql

While fetching records from mysql randomly about 40 - 50% records are getting missed. in the case below I have fetched 10 records only 5 were imported....


schedule => "* * * * *"

statement => "select * from tbl_activity_log where **time** > '2018-06-01' and **time** >= :sql_last_value order by **time** asc limit 10"

	use_column_value => true
	tracking_column => "**time**"
	tracking_column_type => "timestamp"
	clean_run => true 
	last_run_metadata_path => "d:\es\.logstash_jdbc_last_run_test"
}

}

Resolved. I was using time as _id. Bad idea as there can be multiple records at the same time.

 document_id => "%{time}"

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.