I have an Elastic Cloud instance, set to deploy in AWS (v7.13.x). Additionally, I have Logstash (v7.13.4) running within an AWS account, which is in the same region as the Elastic Cloud instance.
In order to get Logstash to post data across to Elastic Cloud privately I have utilised an AWS PrivateLink. I know this link works as I can
curl it from my AWS VPC where Logstash runs and receive back the
/ endpoint result (You know, for search!). Additionally, I can run a dummy pipeline from Logstash to Elasticsearch and the data arrives in Elasticsearch.
My problem is that I cannot seem to get Logstash to show up on the Kibana monitoring page.
I can at least confirm the following:
xpack.monitoring.collection.enabledis set on Elasticsearch as persistent, and the Stack Monitoring page has suddenly activated and now reports Elasticsearch, Kibana and APM+Fleet
- No matter what I do, Logstash seems to never appear on the monitoring page
- Logstash generally can write into Elasticsearch data indexes, which I have tested with a dummy pipeline
- There are no monitoring indexes for Logstash in my Elasticsearch instance
- I can see
.monitoring-kibana-7-<date>indexes but no Logstash one
- I presumed this was due to lack of permissions, but I have since given the user Logstash is running as the
superuserrole, and it still does not appear to happen.
- I can see
I am stumped. Can anyone please advise? I know that the AWS PrivateLink-Elastic Cloud link can only be initiated one way over TCP, and was wondering if the monitoring connection actually runs Elasticsearch->Logstash and is being blocked?
I do not see anything obvious in my logs (which I can post, but I am not sure what I am looking for). Can anyone advise?