How to POST JSON array to Elastic Search

Elastic Stack Elasticsearch
1

I am trying to do a curl call to post my JSON to Elastic search. My JSON file has the following data.

[
{
name1:val1,
name2:val2
},
{
name1:val3,
name2:val4
}
]

Since the JSON starts with array I get a mapping exception when I try to POST The data to ElasticSearch.
ERROR:
{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"failed to parse"}],"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"not_x_content_exception","reason":"Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes"}},"status":400}LM-SJC-11002568

If i change my JSON to the below format, I am able to post the data to elastic search.
{
"result":[
{
name1:val1,
name2:val2
},
{
name1:val3,
name2:val4
}
]
}
But performing analysis on this JSON is not yielding correct results. Because it considers only the "result" while counting the number of documents and hence the "total" is always "1".

My ElasticSearch JSON:
{
"took":16,
"timed_out":false,
"_shards":{
"total":5,
"successful":5,
"failed":0
},
"hits":{
"total":1,
"max_score":1,
"hits":[
{
"_index":"testing",
"_type":"1",
"_id":"******",
"_score":1,
"_source":{
"result":[
{
name1:val1,
name2:val2
},
{
name1:val3,
name2:val4
}
]
}
}

NOTE: Total is one though result has two documents inside it. I need a way to upload the Json Array directly into elasticSearch

2

Please format your code.

A document like:

[
// Something
]

Is not a valid JSON document. You need to provide indeed a JSON document to elasticsearch.

Here you want to index:

{
name1 : val1,
name2 : val2
}

and

{
name1 : val3,
name2 : val4
}

As they are 2 different docs for your use case.

3

I find running JSON objects through 'jq(1)' an easy way to validate them for correctness. You could also use an online tool like sense or JSON lint (note that this might expose your data, so use test data).

4

Thank you for your responses. I have a list of objects in python. I am using json.dumps(list_obj) which gives me a JSON array. I want each array element of the array to be one document in elastic search. Please let me know if there is any way to do this.

5

It's a Python question IMO.

6

I would think that you would hold off on turning your list into an object until you're ready to submit the document, and instead iterate over the list items, dump those, and send the dumped document to elasticsearch.

Pseudocode (where es.index is your favorite way to get the thing indexed; there are Python ES bindings...)

for x in list:
  obj = json.dumps(x)
  es.index(obj)
7

Hi ,
I am also facing the similar challenge.

The data I have looks similar to this -

{"totalName":2,"names":[{"name":"name1"},{"name":"name2"}]}

I understand the approach you proposed works, but is it efficient to make separate REST call for each document when we have an array of more than 10k documents ?

Note - I understand that there is one way ; use Bulk API but I could not find a way to post the Array of JSON Documents via that where each document in array will be treated as a different document.

Thanks in advance .