What's the proper way to set the op_type metadata field?

[jdmcalee (Justin McAleer)]

I am trying to have filebeat use the index operation rather than create, and after digging through the beats source code, I came to this pull request where support was added to set the @metadata.op_type field to index, create, or delete to perform those operations. However, no documentation changes were included in the pull request and no discussion of how to actually use the functionality was either.

After further examining the source code, I determined that only certain processors allow you to correctly add/set metadata fields. If you try to use the add_fields processor, for example, you'll somehow end up with a second @metadata field in your event. Different processors set field values in different ways, and it seems only the event.PutValue method correctly sets values in event's true @metadata field.

All that said, the best working arrangement I have so far is to:

1. use the add_fields processor to add some normal field to the event with the desired value of op_type
2. use the convert processor (it's one that uses event.PutValue) to copy the value from the added field into the @metadata.op_type field (as type string).
3. use the drop_fields processor to remove the field added in step 1.

I am hoping to find a more elegant solution, but I can go with that if necessary.

[jdmcalee (Justin McAleer)]

For posterity's sake, thanks to someone opening this issue there is indeed a better solution:

processors:
  - script:
      lang: javascript
      id: my_filter
      source: >
        function process(event) {
            event.Put("@metadata.op_type", "index");
        }