Logstash Output to Elastic Severless via API Issues

Elastic Stack Logstash
1

Hello,

We are currently moving our Elastic Cloud stack to Elastic Serverless on AWS and having issues trying to output Logstash data up the new to Elastic Serverless. Following documentation, we are providing hosts URL (with port 443 so it does not default to 9200) and the API key that was generated within Elastic Serverless.

Current logstash.yml comfiguration:

path.data: /var/lib/logstash
path.logs: /var/log/logstash

pipeline.ecs_compatibility: v8

xpack.management.enabled: true
xpack.management.pipeline.id: ["our_snmp"]

xpack.management.elasticsearch.hosts: "https://[COMPANY_PROJECT].aws.elastic.cloud:443"
xpack.management.elasticsearch.api_key: "[ID:API_KEY]"

Our Logstash Pipeline with Elastic Serverless:

output {
  elasticsearch {
    hosts => ["https://[COMPANY_PROJECT].aws.elastic.cloud:443"]
    api_key => "[ID:API_KEY]"
    
    data_stream => "true"
    data_stream_type => "metrics"
    data_stream_dataset => "logs"
    data_stream_namespace => "our-snmp-datastream"
  }
}

We are getting several errors in the Logstash logs related to not being able to retrieve Elasticsearch version/cluster info, but the first error seems to be that Logstash is trying to use the API on the Kibana page rather than Elasticsearch:

2024-08-07T16:21:40,278][ERROR][logstash.licensechecker.licensereader] Unable to retrieve Elasticsearch version {:exception=>LogStash::Json::ParserError, :message=>"Unexpected character ('<' (code 60)): expected a valid value (JSON String, Number, Array, Object or token 'null', 'true' or 'false')\n at [Source: (byte[])\"<!DOCTYPE html><html lang=\"en\"><head><meta charSet=\"utf-8\"/><meta http-equiv=\"X-UA-Compatible\" content=\"IE=edge,chrome=1\"/><meta name=\"viewport\" content=\"width=device-width\"/><title>Elastic</title><style>\n        \n        @font-face {\n          font-family: 'Inter';\n          font-style: normal;\n          font-weight: 100;\n          src: url('https://kibana.estccdn.com/a98a8ab1019f/ui/fonts/inter/Inter-Thin.woff2') format('woff2'), url('https://kibana.estccdn.com/a98a8ab1019f/ui/fonts/inter/Inte\"[truncated 187120 bytes]; line: 1, column: 2]"}

Is there a different URL used to connect Logstash and Elasticsearch in the Elastic Serverless project rather than the Kibana pages? If so, where would it be retrievable in Elastic Serverless since there is no way to get the command line?

Also, we have found that Elasticseach is only on version 8.11.x from the Dev Tools (outside of our power to update). Would the version being bellow 8.14.x cause the issue?

Thank you

2

For anyone that happen to come across this post having similar issue, we discovered the issue was the URL. We did not realize the URL we had been using had "kb" with in it since it was actually for accessing the Kibana web page rather than it being part of the project ID. Instead, "es" should have been within the URL so that the API is used to access Elasticsearch instead.

#Old/Wrong URL
https://[COMPANY_PROJECT].kb.us-east-1.aws.elastic.cloud

#New/Correct URL
https://[COMPANY_PROJECT].es.us-east-1.aws.elastic.cloud

Fixing the URL to "es" in both the Logstash Pipeline on the serverless Kibana page and the logstash.yml on the Logstash server allowed the tunnel to be completed.

1 Like