# Logstash duplicate index data

**URL:** https://discuss.elastic.co/t/logstash-duplicate-index-data/124652
**Category:** Elasticsearch
**Created:** [March 19, 2018, 10:11pm UTC](https://discuss.elastic.co/t/logstash-duplicate-index-data/124652 "2018-03-19T22:11:59Z")
**Posts on this page:** 2
**Page:** 1

<div class="post-metadata">

### Author: ![geethu\_priya](https://avatars.discourse-cdn.com/v4/letter/g/71e660/32.png) [@geethu\_priya](https://discuss.elastic.co/u/geethu_priya)
#### Post date: [March 19, 2018, 10:11pm UTC](https://discuss.elastic.co/t/logstash-duplicate-index-data/124652/1 "2018-03-19T22:11:59Z")

</div>

Hi,

Am trying to load 4TB of data into my elastic search server  
Am using java bulk api for one time load, after which am using logstash to poll and update the existing index for any delta update.

But first time after creating index using bulk api and when I run logstash with sql\_last\_value ,it recreates all the documents that already exist in index.

Am using nested structure for my documents

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [April 16, 2018, 10:12pm UTC](https://discuss.elastic.co/t/logstash-duplicate-index-data/124652/2 "2018-04-16T22:12:23Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.