Logstash elastic_app_search output plugin returning 404 page

Elastic Stack Logstash
1

I have used Logstash to pass documents to App Search since 2022, but after a recent Elasticsearch update (to 8.15) my systemctl logs are showing a 404 error, with the html for this page returned in the response:

Screen Shot 2024-11-05 at 5.38.28 PM
Screen Shot 2024-11-05 at 5.38.28 PM1422×778 84.9 KB

This is my current output plugin for app search:

output {
  elastic_app_search {
    url => "https://xxxxxxxx.ent.us-central1.gcp.cloud.es.io/api/as/v1"
    api_key => "xxxxxxxx"

    document_id => "%{video_url}-%{segment}"
    engine => "minutes-2022-09-09"
  }

  # for testing, print to std out
  stdout { codec => rubydebug }
}

I have used the same credentials in a test python script, which does work:

from elastic_app_search import Client

client = Client(
    base_endpoint="xxxxxxxx.ent.us-central1.gcp.cloud.es.io/api/as/v1",
    api_key="xxxxxxxx"
)

engine_name = "minutes-meta"
query = "bikes"
response = client.search(engine_name, query)
print(response)

Which api endpoint and credential set do I need to use now that App Search is under the Kibana umbrella? Why can I access app search via the same API when using the python client, but the Logstash plugin is redirected to the 404 page?

Any help appreciated, thanks!!