Functionbeat does not update documents with existing _id

[amiPorat]

Hello,

I am currently deploying Functionbeat configured to accept events from an SQS queue. The goal is to use a processor/ingest pipeline to extract and set _id from fields in the incoming messages. Documents with existing _id do not appear to be pushed from functionbeat at all. I have attempted to isolate the behavior with the below steps:

• Write message to aws sqs queue i.e.
  aws sqs send-message --queue-url <queue_url> --message-body '{
  "val": "foo",
  }' --message-group-id f1 --message-deduplication-id 1

Doc is successfully written to ES, with _id: <doc_id>.

• Take the ID from the created message above, and write the message to queue again with updated values and pass in the _id.
  aws sqs send-message --queue-url <queue_url> --message-body '{
  "val": "bar",
  "_id": <doc_id>
  }' --message-group-id f1 --message-deduplication-id 2

When I do this I do not see the document being updated or rewritten at all.

Any guidance here would be greatly appreciated. Thank you for your time and help.

Best,
Ami

[shaunak (Shaunak Kashyap)]

Do you see the second document (the one with the _id set being indexed as an entirely new document in Elasticsearch?

[amiPorat]

The second document does not show up at all.

[shaunak (Shaunak Kashyap)]

Hmmm, @kvch any idea why this might be happening? I would've expected the second document to show up as an entirely new document in ES (essentially ignoring the _id field from the message consumed from the queue).

[tzaitsev (Timophey Zaitsev)]

Hi all, any update here? It's been 20 days since we've escalated this and it seems like there is no activity?

Is the recommendation here not to use functionbeat?