Bulk insert API with json file not working

Hi, I have a very large file with like 1 million records or 1,000,000 records (either way its a lot :D)

I am using the curl command:
$ curl -s -H "Content-Type: application/json" -XPOST localhost:9200/_bulk --data-binary @assets-latest.json

The cursor blinks a bit in my terminal (like 2seconds), then just disappears and nothing happens. When I make a POSTMAN API "GET" request, I get 0 hits. Checking on the Kibana console for that specific index, "assets", returns nothing aswell.

It's probably not going to work.

Few things:

  1. does your json file respect the expected BULK specific format (header+content)?
  2. how big is the file (in mb)?

I'd recommend anyway to send smaller bulk requests, like 10000 per 10000 items.

OK thank you. I found that one of our developers here our company wrote a shell script to remap/transform the JSON for a format to be ingested into Elasticsearch.

Thank you for the response.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.