[ingest-pipeline] remove field from target index

[seddikalaouiismaili (sai)]

Hi community,

I'm trying delete some unused fields from index, through the ingest pipeline.

Current config :

        "remove": {
            "if": "ctx._index.contains('stg-index-short-')",
            "field": "message"
        }
    }

Expected behavior :

_index: stg-index-short-*   

should return nothing

Current behavior :

Still having message field on the index stg-index-short-*

There is any another way to delete this field from target ?

Thank you for your help

[leandrojmp (Leandro Pereira)]

I don't think you can use the _index field to filter since it may only exists after all the processing is done.

You will need to use another filter to field.

[seddikalaouiismaili (sai)]

thank you for the reply ..
I tried to filter with another field, but didn't help

    {
        "remove": {
            "if": "lg.meta.component == indexshort",
            "field": "message"
        }
    }

I got this error

Could not Put pipeline stg-pipeline : [{'type': 'script_exception', 'reason': 'compile error', 'processor_type': 'remove', 'script_stack':

[leandrojmp (Leandro Pereira)]

Please share the entire error.

Also, If I'm not wrong the conditional is wrong.

It needs to be something like this: ctx.lg?.meta?.componet == 'indexshort'"

[seddikalaouiismaili (sai)]

Thank you that works fine