Best way to export Index data(10 million) from elastic search to custom json format

Elastic Stack Elasticsearch
1

Sample index data

{
"_index" : "sampleindex",
"_type" : "_doc",
"_id" : "196575730",
"_version" : 2,
"_seq_no" : 2230396,
"_primary_term" : 1,
"found" : true,
"_source" : {
"id" : 19657,
"name" : "wer_1212",
"price" : "1000.0",
"updatedDate" : "2019-06-26T15:30:42.824993-05:00"
}
}

Expected Output format

id|name|price|updateddate
19657|wer_1212|1000.0|06/26/2019

2

I would go with one of this two options, the one you feel more confortable with:
Just with Logstash:
elasticsearch-input using scroll option and output-file
The other option is to use a client library like Python. It provides you an iterator over your query, so it is very memory efficient:
doc-here

1 Like
3

Thanks @Miguel1

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.