# Bulk upload misses few data entries in elastic

**URL:** https://discuss.elastic.co/t/bulk-upload-misses-few-data-entries-in-elastic/131486
**Category:** Elasticsearch
**Created:** [May 11, 2018, 1:55pm UTC](https://discuss.elastic.co/t/bulk-upload-misses-few-data-entries-in-elastic/131486 "2018-05-11T13:55:17Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![Dhivya\_Buvanesh](https://avatars.discourse-cdn.com/v4/letter/d/ad7895/32.png) [@Dhivya\_Buvanesh](https://discuss.elastic.co/u/Dhivya_Buvanesh)
#### Post date: [May 11, 2018, 1:55pm UTC](https://discuss.elastic.co/t/bulk-upload-misses-few-data-entries-in-elastic/131486/1 "2018-05-11T13:55:17Z")

</div>

We are trying to upload packet captures files (tshark -r capture.pcap -T ek \> packets.json) , randomly data are getting missed to upload while using BULK upload to elastic

curl -s -H "Content-Type: application/x-ndjson" -XPOST "localhost:9200/\_bulk" --data-binary "@packets.json”

There are 25,000 data entries totaly in that JSON, on upload we see only around 24445 data

No errors are displayed while uploading to elastic, Request for help, on how we can resolve this issue?

---

<div class="post-metadata">

### Author: ![thiago](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thiago/32/32096_2.png) [@thiago](https://discuss.elastic.co/u/thiago)
#### Post date: [May 16, 2018, 3:59am UTC](https://discuss.elastic.co/t/bulk-upload-misses-few-data-entries-in-elastic/131486/2 "2018-05-16T03:59:00Z")

</div>

How are you processing this `packets.json` into the supported BULK format? Are you manually generating an ID for each document? If so, maybe there are repeated IDs?

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 13, 2018, 3:59am UTC](https://discuss.elastic.co/t/bulk-upload-misses-few-data-entries-in-elastic/131486/3 "2018-06-13T03:59:04Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
