Datastream with upsert

[djkprojects]

Hello,

We are pulling data from MS SQL database into Elastic via Logstash which is working fine however some records get updated and so we want to update the existing entries in Elastic accordingly.

The data is stored as a datastream but it looks that these don't support doc_as_upsert and action parameters in elasticsearch output.

We can't find any documentation that confirms that so the question is are we missing something here or it is not supported? If the latter how can we go about it using logstash?

[leandrojmp (Leandro Pereira)]

@djkprojects

We can't find any documentation that confirms that so the question is are we missing something here or it is not supported?

Data streams are append-only, this is described in the documentation about data streams, so they do not support doc_as_upsert, the only action from logstash that it supports is create .

You will need to use normal indices with a rollover alias as explained in this documentation or use time-based indices.