Change Index Ingestion method

[RodAndTom (Rodrigo Tomaz)]

Hi,

I finally managed to setup a MongoDB connector integration with Elastic through Kibana, but it create a index and I can't change the settings and mappings for this Index (Ingestion method is shown as "Connector".
So I tried to manually create a index with all the settings and mappings I want (Ingestion method is shown as "API", but I can't associate this index to my MongoDB connector

Is there a way to update the settings and mappings for a Index created for a connector, or change the index, to use one created manually?

[RodAndTom (Rodrigo Tomaz)]

I managed to update the index created with the connector (by closing the index before updating the settings and mappings).
However, I see no option tho choose the MongoDB columns and the respective ES fields. Do I need to use a ingest pipeline to do it?

[Dana_Juratoni]

@RodAndTom Thanks for reaching out!

We are actively working on better index management for ingestion methods, in the meantime you landed on the solution we would have suggested as well - changing index mapping and setting before starting to ingest data.

There are several ways available to transform data when using Elastic connectors, you can read more about extract and transform general concepts here and data filtering concepts here.

For MongoDB in particular, you can use Advanced Sync Rules e.g. find queries or aggregation pipelines to manipulate the documents at data source level. This advanced filtering option transforms data before it reaches Elasticsearch.
Alternatively, ingest pipelines are another valid option to manipulate data before indexing it.

Hope that helps!