I am writing a script processor which will retain only the relevant fields and remove all the other fields . Here 'message','custom_field','@timestamp','_index','_id','_version','index_name','tags', is getting retained but the fields like 'audit.log.0','audit.log.1','event.type','auditd.log.key','event.dataset','event.created','event.original','event.ingested' is not getting retained .
Please provide me a solution . Also i dont want to use remove processor and logstash , I want to handle this using ingest pipeline only since I am using filebeats audit ingest pipeline for parsing the events
{
"script": {
"source": """
ctx.keySet().retainAll(['message','custom_field','@timestamp','_index','_id','_version','index_name','tags','audit.log.0','audit.log.1','event.type','auditd.log.key','event.dataset','event.created','event.original','event.ingested'])
"""
}
}