Hello,
I am trying to send an entire txt file line in one single field to elastic. I am trying to do this using logstash aggregate, but it seems that it is no longer working, each line is being sent as a separate field rather than just one field for the entire file.
Here is the logstach filter that we are using
filter {
aggregate {
task_id => "%{app.code}"
code => '
map["full_message"] ||= ""
map["full_message"] << (event.get("message") + "\n")
event.cancel()
'
push_previous_map_as_event => true
timeout => 60
}
}
with the following logstash.yml configuration
pipeline.batch.size: 125
pipeline.batch.delay: 5
pipeline.workers: 1
http.host: "0.0.0.0"
http.port: 9600
#path.config: "/etc/logstash/conf.d/default-pipeline.conf"
path.data: "/app/logstash/data"
queue.type: persisted
The logstash version we are using is 7.9.1
Any advice on what we are doing wrong?