Logstash aggregate not working as expected

Hello,
I am trying to send an entire txt file line in one single field to elastic. I am trying to do this using logstash aggregate, but it seems that it is no longer working, each line is being sent as a separate field rather than just one field for the entire file.

Here is the logstach filter that we are using

filter {
  aggregate {
    task_id => "%{app.code}"
    code => '
      map["full_message"] ||= ""
      map["full_message"] << (event.get("message") + "\n")
      event.cancel()
    '
    push_previous_map_as_event => true
    timeout => 60
  }
}

with the following logstash.yml configuration

pipeline.batch.size: 125
pipeline.batch.delay: 5
pipeline.workers: 1
http.host: "0.0.0.0"
http.port: 9600
#path.config: "/etc/logstash/conf.d/default-pipeline.conf"
path.data: "/app/logstash/data"
queue.type: persisted

The logstash version we are using is 7.9.1

Any advice on what we are doing wrong?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.