I have a project to send postgresql logs to ELK.
i have tried using a setup to parse through postgres logs and send them to elasticsearch until i saw filebeat can now support postgresql logs.
now there are concerns that postgresql logging may log queries which are problematic and that may contain sensitive data as insert/update params.
i saw this extension (in ruby but there is a similar one in go) that can mask params in the query.
i guess sometime later, this module will have to anonymize the query params for the sake of security/GDPR etc.
can this be done at filebeat level?
i even thought of doing it at the logstash level, but looks like jruby does not like c extensions
The Go library looks good for this task. Can you open an Enhancement Request with this idea? I'd like someone from the infrastructure functional area to review this library and see if there is a problem integrating it with filebeat.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.