Functionbeat does not update documents with existing _id


I am currently deploying Functionbeat configured to accept events from an SQS queue. The goal is to use a processor/ingest pipeline to extract and set _id from fields in the incoming messages. Documents with existing _id do not appear to be pushed from functionbeat at all. I have attempted to isolate the behavior with the below steps:

  • Write message to aws sqs queue i.e.
    aws sqs send-message --queue-url <queue_url> --message-body '{
    "val": "foo",
    }' --message-group-id f1 --message-deduplication-id 1

Doc is successfully written to ES, with _id: <doc_id>.

  • Take the ID from the created message above, and write the message to queue again with updated values and pass in the _id.
    aws sqs send-message --queue-url <queue_url> --message-body '{
    "val": "bar",
    "_id": <doc_id>
    }' --message-group-id f1 --message-deduplication-id 2

When I do this I do not see the document being updated or rewritten at all.

Any guidance here would be greatly appreciated. Thank you for your time and help.


1 Like

Do you see the second document (the one with the _id set being indexed as an entirely new document in Elasticsearch?

The second document does not show up at all.

1 Like

Hmmm, @kvch any idea why this might be happening? I would've expected the second document to show up as an entirely new document in ES (essentially ignoring the _id field from the message consumed from the queue).

Hi all, any update here? It's been 20 days since we've escalated this and it seems like there is no activity?

Is the recommendation here not to use functionbeat?

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.