Duplicate records in Elasticsearch

Try using a fingerprint filter to set the document id. See this thread.