Ingest pipeline routing documents to appropriate target index requires permissions on target index

[leandrojmp (Leandro Pereira)]

@Jurgen_Wagner_DVT

In my opinion, however, this should not be necessary: if a user has been granted access to the virtual index, and the ingestion pipeline internally determines the target index, there should not have to be additional permission checks for that user on the target index.

The indexing of the document happens after the ingest pipeline is executed, and the indexing will be done by same authenticated user that made the request, the ingest pipeline does not change the user, that's why it needs to have permissions to the final target index.

It is not clear how you are changing the value of the _index field to change the target index, but the reroute processor explicitly says that.

Note that the client needs to have permissions to the final target. Otherwise, the document will be rejected with a security exception

@Jurgen_Wagner_DVT

The only way I can imagine solving this issue is by introducing an ingestion pipeline for all target indexes that would prevent direct feeding for documents not having passed through the pipeline of the virtual index.

If you don't trust your data-feeding users in this case, don't give them access to write directly into Elasticsearch. You can put Logstash in front of it, configure an http input and then create your logic to index the data into the correct index.

In this case your users wouldn't even need permission to write into Elasticsearch, just permission to send logs to Logstash, the user configured in Logstash would have the permissions to write in your indices.