Check if value is 'null' So it Can Be Used in a Fingerprint and Document _id

You will note that is my very long thread ........ Did you read it all :slight_smile:

So this is why context is important and just snippets of codes and questions can lead to long winding discussions that are wasteful etc ... example you did not show the config for decode_json_fields that matters...

You need to read the thread from this part from here down

Basically updating documents in data streams does not work, you will need to use indices, not data streams which is OK (sort of lots of limitations) but definitely harder... and has limitations

I would suggest you closely read that thread.... the partial solution with filebeat for 8.6.0 should be here... you will have to use indices not data streams which adds more complexity...

I am happy to help but I can not really help if you do not provide me with complete and / or meaningful configs and data (yes you can sanitize the fields") because to help I have to build samples and complete configs and guess at things when you leave out items it.

So your use case you want to read time series data and update documents... can be done but not easy... and there are pretty severe limitations you can not use ILM for time series data because the updates won't work unless you know the correct data so you are limited to a single index... but we will leave that for another time....

Read the thread closely and I will get your processors to work, your syntax is not correct :slight_smile: