Short story: I'm using the es.js client creation code for the api. I call bulk with the api. I get a request timeout. I'm passing a newline delimited json array.
Longer story (read the short story first, please):
My code is:
var client = new elasticsearch.Client({
host: {
protocol: 'http',
host: 'myhost.com',
path: '/myindex/_doc'
},
log: ['error', 'trace'],
sniffOnStart: false,
requestTimeout: 110000,
pingTimeout: 110000,
createNodeAgent(connection, config) { // TODO: bug, connection doesn't stay alive "no living connection"
return new AgentKeepAlive(connection.makeAgentConfig(config));
}
});
My bulk code:
client.bulk({
body: out
}, function (err, resp) {
console.log('in');
console.log(err);
console.log(resp);
});
The out object is (abbrev):
[
'{"index":{"... }}
'{"myname":"value"}'
]
I do wonder if the single quotes around each array element is problematic, BUT, I'm using AWS, and AWS shows the ES trace:
The ES Trace is written to the log after the client.bulk request timeout error... maybe a timing issue with async? I dunno... I'm grasping for an explanation.
Update 2: If my hostname includes a .com (myhost.com) nothing happens... not even a client.ping shows it's been attempted. If I remove the .com (myhost) from the hostname, the ping and bulk make an attempt but there is a no living connection warning.
Why? I'm using an AWS endpoint, so I kind of need that .com, don't I?
Cloud by elastic is the only way to have access to X-Pack. Think about what is there yet like Security, Monitoring, Reporting and what is coming like Canvas, SQL...
I am using ES hosted on AWS. However, I'm using the Elastic elasticsearch js API. That belongs to Elastic? An endpoint is an endpoint, right? It's almost like there is code that sees a competing cloud service and blocks the connection. My code connects to localhost:9200 but not an AWS endpoint. Conspiracy theory, maybe. Also, I don't think I need x-pack.
As far as I know AWS ES requires authentication, and I do not believe it supports HTTP auth and role-based access control like Elastic Cloud does. For Logstash they therefore developed a special Elasticsearch plugin that handles this. As I have no experience with AWS ES service I can not tell whether it is possible to configure elasticsearch.js to connect to it or not.
I you do not get any response from other users here I would recommend you reaching out to AWS support as they should know exactly what is required.
I was being facetious... my coping mechanism for this frustration. I know i'm super close and I've researched this for a bit. I'm probably a brain fart away from figuring it out... or finding the npm package to use where someone else figured it out for me.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.