Replace existing record if _id matches (filebeat)

Elastic Stack Elasticsearch
1

Hello,
Lets say I have 3 records with same correlation id. The only thing that is different is statuses of these records (active, errornous, issued). I want the flow to be like that:
Example:

  1. 1000 records are generated sent to elasticsearch via filebeat
  2. Each record's _id=CorrelationId
  3. Another 1000 records are generated.
  4. If there are records with existing correlation id in the index I want them to be replaced (or updated if you can call so).

So in order to achieve that I have 2 possible solutions that are only assumptions:

  1. Keep all records without replacing and filter them with query in kibana (is it possible?)
  2. Replace old records with the new ones. (Maybe filebeat automatically overwrites records when it sees that _id already exists)

I would be happy if you could provide me with some guidances how can I achieve that? I have been searching for the solution for quite a long time.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.