# Missing some docs when using bulkprocessor! Is it a bug?
**URL:** https://discuss.elastic.co/t/missing-some-docs-when-using-bulkprocessor-is-it-a-bug/51257
**Category:** Elasticsearch
**Created:** [May 30, 2016, 12:44am UTC](https://discuss.elastic.co/t/missing-some-docs-when-using-bulkprocessor-is-it-a-bug/51257 "2016-05-30T00:44:12Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@Yongyao\_Jiang](https://discuss.elastic.co/u/Yongyao_Jiang)
#### Post date: [May 30, 2016, 12:44am UTC](https://discuss.elastic.co/t/missing-some-docs-when-using-bulkprocessor-is-it-a-bug/51257/1 "2016-05-30T00:44:13Z")
I was using Bulkprocessor to index some documents (1005), but I only ended up with 1003 in ES. Also, this value could change when I change the settings of bulkprocessor (setConcurrentRequests to 10). Can someone help me out? Thanks. Below is my code,
bulkProcessor = BulkProcessor.builder(
...)
.setBulkActions(1000)
.setBulkSize(new ByteSizeValue(1, [ByteSizeUnit.GB](http://ByteSizeUnit.GB)))
.setConcurrentRequests(1)
.build();
IndexRequest ir = new IndexRequest(config.get("indexName"), outputType).source(
jsonBuilder().startObject()
.field("SessionID", entry.getKey()).field("SessionURL", sessionURL)
.endObject());
es.bulkProcessor.add(ir);
---
### Author:  [@nik9000](https://discuss.elastic.co/u/nik9000)
#### Post date: [May 30, 2016, 1:11am UTC](https://discuss.elastic.co/t/missing-some-docs-when-using-bulkprocessor-is-it-a-bug/51257/2 "2016-05-30T01:11:54Z")
How did you get the numbers? Like are you counting the numbers in the
listener or are you counting after you shut down the processor? If you are
counting after, are you refreshing? That is what I'd check first.
After that I'd try to make a minimal recreation. Maybe that'd find a
mistake you made or, if not, it'd be way easier to work from for us to
reproduce the bug.
---
### Author:  [@Yongyao\_Jiang](https://discuss.elastic.co/u/Yongyao_Jiang)
#### Post date: [May 30, 2016, 1:43am UTC](https://discuss.elastic.co/t/missing-some-docs-when-using-bulkprocessor-is-it-a-bug/51257/3 "2016-05-30T01:43:56Z")
Thanks! The original number was counted in the loop that I used to add indexRequest to Bulkprocessor. The result number was counted in Sense after I shut down both the bulkprocessor and node.
bulkProcessor.awaitClose(20, TimeUnit.MINUTES);
node.client().admin().indices().prepareRefresh().execute().actionGet();
node.close();
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [May 30, 2016, 1:48am UTC](https://discuss.elastic.co/t/missing-some-docs-when-using-bulkprocessor-is-it-a-bug/51257/4 "2016-05-30T01:48:50Z")
Do you flush the bulk processor ?
Because here, bulk processor send request to Elasticsearch every 1000 docs or every 1gb.
I personally use a flush every 5 seconds with `setFlushInterval`.
---
### Author:  [@Yongyao\_Jiang](https://discuss.elastic.co/u/Yongyao_Jiang)
#### Post date: [May 30, 2016, 2:08am UTC](https://discuss.elastic.co/t/missing-some-docs-when-using-bulkprocessor-is-it-a-bug/51257/5 "2016-05-30T02:08:11Z")
Good point. I should try that. But the reason I didn't use flush before is that I have two concerns.
1. I think bulkProcessor.awaitClose(20, TimeUnit.MINUTES) would flush all remaining docs by itself, according to the documentation.
2. Also, does setFlushInterval sort of conflict with setBulkSize() and SetBuilActions()?
Thanks,
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 5, 2017, 10:48pm UTC](https://discuss.elastic.co/t/missing-some-docs-when-using-bulkprocessor-is-it-a-bug/51257/6 "2017-07-05T22:48:13Z")