HI,
I'm very new to the elastic world, but I was able to get logstash setup and successfully moving data from a SQL Server into my index in elasticsearch using the JDBC driver setup. As of now, my ETL process is currently scheduled to run every minute as my site will be pretty transactional in nature.
When I query (via REST) my elasticsearch index by the document Id ("__id"), I can see the "version" key is incrementing by 1 every time . I'm assuming what's happening is the document is being overwritten despite no changes to the data.
Is there a way I can only have the "__version" key increment by 1 if the record/document has actually changed?
Here is the way my output config file looks, not sure if I need to modify this in any way to achieve my result.
output {
elasticsearch {
action => "update"
hosts => ["localhost:9200"]
index => "company"
document_type => "article"
document_id => "%{article_id}"
doc_as_upsert => true
}
stdout { codec => json_lines }
}