Hey guys;
I'm processing records from DynamoDB via a function beat and it's working great, if I specify the source record as the "document_id" for the "decode_json_fields" process Elastic correctly uses this as the ID for the resulting document in ES.
But what I notice is that when that same record in the source database is updated and it passes through the function beat again ES doesn't seem to be doing an upsert. I don't get a dupe record which is good, but ES doesn't reflect the updated fields.
Any pointers greatly appreciated.
Thanks!
Reference: