Filebeat 'replace' processor cant be empty

Elastic Stack Beats
1

Hey, I just upgraded to filebeat 8.15.0 and now my filebeat config isn´t working anymore.

If the replacement is an empty string, filebeat wont start.

      - replace:
          fields:
            - field: "host.hostname"
              pattern: ".my.domain"
              replacement: ""

I get the following error:

{"log.level":"error","@timestamp":"2024-09-03T05:43:17.517Z","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.handleError","file.name":"instance/beat.go","file.line":1349},"message":"Exiting: error initializing processors: failed to unpack the replace configuration: string value is not set accessing 'processors.34.replace.fields.0.replacement' (source:'/etc/filebeat.yml')","service.name":"filebeat","ecs.version":"1.6.0"}

Is there another way I can replace/delete a part of a string in my host.hostname field? Or is this behaviour a bug?

Edit: I changed my replacement from an empty string to a space, but this is a bad workaround because in Kibana I need to add this space after my host.hostname otherwise my filter doesn´t work.

Edit 2: I use the "Script Processor" now, as long as I don´t know if this is a bug or not. Here is my code if anything is facing a similar issue:

      - script:
          lang: javascript
          source: >
            function process(event) {
              var hostname = event.Get("host.hostname");
              var domain = '.my.domain';
              var updatedHostname = hostname.slice(0, -domain.length);
              event.Put("host.hostname", updatedHostname);
              return event;
            }
2

This feels like an issue with the yaml parser from elastic-agent to filebeat.
We faced the same issue updating from 8.14.0 to 8.15.0.

Moreover, when trying to use the script processor, unless we call it from a file, we got the error [elastic_agent.filebeat][error] Input 'udp' failed with: failed to unpack the replace configuration: string value is not set accessing 'processors.7.replace.fields.0.replacement' no matter what (we tried inline, with " and ', as well as using | instead of > to pass the code.

Upon checking the elastic-agent yaml configuration with elastic-agent inspect, it removed the leading and trailing quotes from the script parameter, which when used with special characters could break the yaml interpretation.

Here is the script, called from a file:

function process(event) {
  var original = event.Get("message");
  var original_clean = original.replace(/["]/g, "");
  event.Put("event.original", original_clean);
  event.Delete("message")
  return event;
}

Even trying with the first example from the documentation we got the same error.

It would be nice if someone from Elastic could verify if there is indeed a bug.