Bulk Index newline termination error from Elastic cloud

I am sending content-Type/x-ndjson from AWS Nodejs lambda functions to elastic cloud cluster using https and JSON.Stringify to create message that looks like this

"{ "index" : { "_index" : "requests-2018-08", "_type" : "_doc"}}\n{"timestamp": "2018-08-27T22:29:14.583Z","name":"tour-api-dev-tourcmsAvailability","account":"832247244038","request-id":"a0872a41-aa48-11e8-b5ee-0902424d18a4","msg":"[ERROR] SyntaxError: Unexpected token u in JSON at position 0 at Object.parse (native) at module.exports.handler (/var/task/src/tourcms/availability/get.js:181:27)"}\n{ "index" : { "_index" : "requests-2018-08", "_type" : "_doc"}}\n{"timestamp": "2018-08-27T22:29:14.667Z","name":"tour-api-dev-tourcmsAvailability","account":"832247244038","request-id":"a0872a41-aa48-11e8-b5ee-0902424d18a4","msg":"REPORT RequestId: a0872a41-aa48-11e8-b5ee-0902424d18a4Duration: 348.23 msBilled Duration: 400 ms Memory Size: 128 MBMax Memory Used: 28 MB"}\n"

I am not pretty printing the message with JSON.stringify just sending
and getting this error:

{
"error": {
"root_cause": [
{
"type": "illegal_argument_exception",
"reason": "The bulk request must be terminated by a newline [\n]"
}
],
"type": "illegal_argument_exception",
"reason": "The bulk request must be terminated by a newline [\n]"
},
"status": 400
}

To me it looks like my message is terminated with a newline and when I add one more line break I get a metadata action failure as there is no action on the next blank line.

It looks like the meta characters in your payload does not get translated into newlines. If you instead used the elasticsearch.js client, it would handle this for you.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.