Split string by newline to Logstash

[2021-07-26 12:09:19.877 +08] [pgAdmin 4 - DB:testdatabase] [postgres]-[testdatabase]-[::1(57223)] [] [00000]LOG:  disconnection: session time: 0:00:00.218 user=postgres database=testdatabase host=::1 port=57223
[2021-07-26 12:09:53.663 +08] [pgAdmin 4 - CONN:1699206] [postgres]-[testdatabase]-[::1(53775)] [] [00000]LOG:  AUDIT: OBJECT,1,1,READ,SELECT,TABLE,public.testjoe,select * from testjoe;,<none>
[2021-07-26 12:56:35.173 +08] [pgAdmin 4 - DB:testdatabase] [postgres]-[testdatabase]-[::1(57224)] [] [57P01]FATAL:  terminating connection due to administrator command
[2021-07-26 12:56:35.173 +08] [pgAdmin 4 - DB:postgres] [postgres]-[postgres]-[::1(57218)] [] [57P01]FATAL:  terminating connection due to administrator command

My filebeat capture the above 4 lines of string with \n, newline, and pass it to logstash to spilt them into different fields then insert into Elasticsearch. How can I ask filebeat to send 1 line at a time to logstash so that I can have 4 different records/document in Elasticsearch? I wanted to have separate document for every line of string (log).

What does your filebeat configuration look like?

I only enabled the postgresql.yml module in filebeat.

- module: postgresql
  log:
    enabled: true
    var.paths: ["C:/Program Files/PostgreSQL/13/data/log/*.log"]

In filebeat.yml file, I have removed the output to elastiscearch and enable the output to logstash. The rest remains as the default.

output.logstash:
hosts: ["localhost:5044"]

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.