I use elasticsearch with fluentd and kibana as a logserver. Now I have the problem that I need to delete or pseudonomize logs after a certain time because of data
I.e.: all logs sent from dovecot should be kept only for 7 days and all apache-logs should get their IPs pseudonomized after 24 hours because of data protection laws. General system logs can be kept indefinitely. How can I realize something like that with elasticsearch?
Use time-based indices and place data with different retention requirements in different indices. In order to change data after a certain time you will need to reindex, e.g. by using the update by query API, potentially together with an ingest node pipeline to handle the transformation.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.