If exactly the same documents are missing from two indices where one uses autogenerated IDs and the documents can be inserted later without mapping conflicts, I do not think the issue necessarily is in Elasticsearch. If you add another output and write the unique IDs to a file you should be able to verify that Logstash actually processes all the data.