Remove pipeline using field wildcard

Elastic Stack Elasticsearch
1

I want to remove a field starting with ElapsedTimeAt* in my remove processor pipeline. Is it possible?

2

Reading the code, I don't think it is possible to use wildcards.

1 Like
3

Thanks for the quick reply. This is what i am trying to achieve.

I have a incoming document and in Logstash i add a field for which the name is dynamic (filed name will be ElapsedTimeAt_timestamp where timestamp will be substituted with @timestamp value). This field i need to make it available in one index and should be removed before indexing into other index. Struggling to implement this. Is there any way to do this ??

4

May using an ingest script processor?

1 Like
5

Can i have something like
ctx.ElapsedTimeAt_%{timestamp} ? Will it compute the column name dynamically ?

If you have any example that would help me great ..

6

Yes you can have something similar to this if there is a field which contains the value you are looking for. See https://www.elastic.co/guide/en/elasticsearch/reference/6.6/accessing-data-in-pipelines.html#accessing-template-fields

For Ingest Script Processor, read https://www.elastic.co/guide/en/elasticsearch/reference/6.6/script-processor.html. I'm not a painless expert so I can't really tell how to read all the fields from your documents and compare to a regex/wildcard and remove it or not.

1 Like
7

Same here David.... very beginner in painless ... But will figure it out and update you if i am successful... BTW thanks much for the quick reply.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.