How to bulk insert 600MB large json file to elasticsearch?

[Ash_Chamari]

How to bulk insert 600MB large json file to elasticsearch?

[Christian_Dahlqvist]

Have a look at these recent threads:

1GB single json record

Hi all, We are trying to let the filebeat to collect a json file. Which is a single record json file, but it got 1GB file size. We tried filebeat send to logstash also the filebeat send to ES directly. Unfortunately we are not lucky enough. Does any body face this problem before? Any replies will be very helpful. Thanks, From Mulder

How to bulk load huge JSON docs from files using Java client?

Hi. I need your help / advice / opinion: TL;DR: How do I read file from the disk for bulk loading using IndexRequest.source(BytesReference)? And the detailed question: The context: I am indexing text in a public dataset, such as: http://data.cityofnewyork.us/resource/nc67-uf89 The dataset includes some simple meta-data fields (such as dataset name and description) and a large CSV table of the data itself. I am indexing the text from the big CSV table. The CSV table size may be a few GBs …

If the file consists of multiple documents, break it up into smaller chunks. If it is a single document, rethink how you are going to index and search it.

[Ash_Chamari]

Hi @Christian_Dahlqvist,

Thanks for your reply.

In the geoJson file, under features array, I have objects that contains geometry and properties and geometry object stores the coordinates.

Currently I am mapping each {geometry & properties} as one document.

Any idea how to index this properly or chunk out to smaller potions?