I have 6374 json documents that I would like to ingest.
- I used the curl command as follows:
curl -H 'Content-Type: application/x-ndjson' -XPOST 'localhost:9200/projects/_bulk?pretty' --data-binary @projects.json
I found that only 1014 documents got ingested as shown by
- Then I thought it could be some issue with the data, and tried to ingest
half of the data (3200 documents)
This time I found that only around 465 documents were ingested.
- I tried again to narrow down the problem by ingesting the first 500 documents.
This time I found only 74 documents were ingested.
An interesting pattern I find is that only 15% of the total amount that I am trying to ingest gets into elasticsearch.
Makes me wonder if there are any thresholds or upper limits that I am exceeding in curl or elasticsearch.
Note that I tried both with and without pretty option. Same results.
I would appreciate any inputs. Thanks!