Ingest pipeline for App Search document indexing

[scottroot (Scott)]

I am using Elastic Cloud Enterprise Search/App Search and indexing json documents via the App Search api. I am not able to get Ingest Pipelines working though.

API endpoint I'm using to index (replaced project/engine names):
'https://myproject.ent.us-central.gcp.cloud.es.io/api/as/v1/engines/myengine/documents'

I have read this topic about custom logs but it doesn't seem to apply when using App Search: Custom Logs Ingest Pipeline - #6 by Doommius

I created a pipeline, tested it and it works. In the index settings (.ent-search-engine-documents-myengine) I tried setting index.default_pipeline and "index.final_pipeline": to "logs-myengine-default" but neither resulted in any difference.

Is there any documentation on using Ingest Pipelines with App Search?
Thanks in adavance

[Sean_Story]

Hi @scottroot,

In versions <= 8.2, App Search makes no use of Ingest Pipelines, and it is heavily discouraged to modify indices and their mappings/settings managed by App Search, or you will risk breaking your ability to migrate/upgrade your deployment.

In 8.3, App Search uses a single ingest pipeline, only for App Search Web Crawler engines, and only for documents that are binary (non-html) in nature. See: Web crawler reference | Elastic App Search Documentation [8.3] | Elastic

In future versions we hope to make this easier and to expand our usages of ingest pipelines in App Search.

In the mean time, you may want to consider using a pure Elasticsearch Index + Ingest Pipeline, and then point App Search to that Elasticsearch Index. See: Elasticsearch index engines (technical preview) | Elastic App Search Documentation [8.3] | Elastic