Hi,
Running Filebeat 7.12 in Kubernetes, I'm unable to configure the postgresql module to parse CSV logs through hints.
I added the following annotations ( logreaper
is the sidecar container that outputs postgres csv logs to stdout):
$ kubectl get pod postgres-cluster-0 -o yaml
apiVersion: v1
kind: Pod
metadata:
annotations:
co.elastic.logs.logreaper/fileset: csv
co.elastic.logs.logreaper/module: postgresql
But the events are still marked as part of the "log" fileset and the whole CSV line appears in the message
field:
{
"_source": {
"@version": "1",
"service": {
"type": "postgresql"
},
"fileset": {
"name": "log"
},
"tags": [
"beats_input_codec_plain_applied"
],
"message": "2021-04-23 16:08:18.554 UTC,,,5483,\"[local]\",6082f0f2.156b,1,\"\",2021-04-23 16:08:18 UTC,,0,LOG,00000,\"connection received: host=[local]\",,,,,,,,,\"\"",
"agent": {
"ephemeral_id": "a4042fe1-d337-4e57-9b6b-8fc3d8b26326",
"name": "k3osworker03",
"hostname": "k3osworker03",
"type": "filebeat",
"version": "7.12.0",
"id": "f8fdf2d7-f87c-4a08-9f94-0edf2570133e"
},
"host": {
"name": "k3osworker03"
},
"ecs": {
"version": "1.8.0"
},
"event": {
"dataset": "postgresql.log",
"module": "postgresql"
},
"@timestamp": "2021-04-23T16:08:18.726Z",
"input": {
"type": "container"
},
"stream": "stdout"
}
}
Am I missing something ?