Bulk upload folder of json

Elastic Stack Elasticsearch
#1

Apologies if this is out there, but I can't seem to figure it out. I have a folder called 'data' that consists of 10,000 geo json files.

I want to be able to upload the entire folder at once. I've tried
curl -XPOST "http://localhost:9200/customer/customer/1/_bulk" --data-binary "@data"

But this gives me:

Warning: Couldn't read data from file "data", this makes an empty
Warning: POST.
No handler found for uri [/customer/customer/1/_bulk] and method [POST]
#2

You won't be able to do that, you will need to concatenate the files together and make sure they have the correct bulk format.

It'd be easier to just do a for loop :slight_smile:

#3

ok so next step, figure out how to loop over my files. Could I do a python loop over the folder and then have the ".json" file go in right after the @ symbol? Or, is there a better method. I know of things like pycurl?

#4

I'd use bash personally, but whatever you want!

#5

Ok Thanks for suggestion, I'm new to this, so will look into using bash, appreciate it

#6

You could use Logstash as well :slight_smile:

#7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.