You only provided snippets of logs so it's very hard for me to tell / help.
You also didn't show me a sanitized version of what the publisher is actually publishing?
You show this ... But leave out all the interesting stuff 
DEBUG [processors] processing/processors.go:203 Publish event: {
I would remove the pipeline, run with -d "*"
Does it look correctly dissected?
Do you see the _id set?
Here is my sample... it works,
It creates a fingerprint
It assigns the _id and when written to elasticsearch uses that _id
- type: filestream
# Unique ID among all inputs, an ID is required.
id: my-filestream-id
# Change to true to enable this input configuration.
enabled: true
# Paths that should be crawled and fetched. Glob based paths.
paths:
- "/Users/sbrown/workspace/customers/acme/sample-data/catalina.out"
processors:
- fingerprint:
fields: ["message"]
target_field: "@metadata._id"
method: "sha1"
And when I run
filebeat -e -d "*"
I see this in the below in the filebeat console... (you have to format it a bit) this is what is published to elasticsearch, it has the _id set.
{
"@timestamp": "2022-12-10T16:24:41.400Z",
"@metadata": {
"beat": "filebeat",
"type": "_doc",
"version": "8.4.3",
"_id": "6d084d6706c79dfb95ff164a9bf6abb0661ed5ee" <!---- Yup _id Set to fingerprint)
},
"message": "2022-11-09 01:26:17 INFO Interesting Log Message",
"input": {
"type": "filestream"
},
"host": {
"ip": [
So if you do not see this ... nothing down stream will work.
I suspect one of the processors it failing for you.