Bulk Index newline termination error from Elastic cloud

Elastic Stack Elasticsearch
1

I am sending content-Type/x-ndjson from AWS Nodejs lambda functions to elastic cloud cluster using https and JSON.Stringify to create message that looks like this

"{ "index" : { "_index" : "requests-2018-08", "_type" : "_doc"}}\n{"timestamp": "2018-08-27T22:29:14.583Z","name":"tour-api-dev-tourcmsAvailability","account":"832247244038","request-id":"a0872a41-aa48-11e8-b5ee-0902424d18a4","msg":"[ERROR] SyntaxError: Unexpected token u in JSON at position 0 at Object.parse (native) at module.exports.handler (/var/task/src/tourcms/availability/get.js:181:27)"}\n{ "index" : { "_index" : "requests-2018-08", "_type" : "_doc"}}\n{"timestamp": "2018-08-27T22:29:14.667Z","name":"tour-api-dev-tourcmsAvailability","account":"832247244038","request-id":"a0872a41-aa48-11e8-b5ee-0902424d18a4","msg":"REPORT RequestId: a0872a41-aa48-11e8-b5ee-0902424d18a4Duration: 348.23 msBilled Duration: 400 ms Memory Size: 128 MBMax Memory Used: 28 MB"}\n"

I am not pretty printing the message with JSON.stringify just sending
and getting this error:

{
"error": {
"root_cause": [
{
"type": "illegal_argument_exception",
"reason": "The bulk request must be terminated by a newline [\n]"
}
],
"type": "illegal_argument_exception",
"reason": "The bulk request must be terminated by a newline [\n]"
},
"status": 400
}

To me it looks like my message is terminated with a newline and when I add one more line break I get a metadata action failure as there is no action on the next blank line.

2

It looks like the meta characters in your payload does not get translated into newlines. If you instead used the elasticsearch.js client, it would handle this for you.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.