Change field name during index

Elastic Stack Elasticsearch
1

It is possible to create a character map analyzer for fields that are indexed but is it possible to do the same for field names?

Example field name would be "A|B" and the need would be to change the pipe to some other character, e.g. underscore "A_B".

2

That's exactly what the Gsub processor does.

Edit: the mutate/Gsub filter documentation from Logstash is more helpful I think, same basic thing.

3

This seems to do whats needed, however it would need to apply to any field and it's not possible for me to list all field names.

4

Do you want to replace all instances of the pipe character with an underscore, or only if they're in the field name?

You can still run a custom normalizer/char filter, just apply it to the _source field and it'll replace all the pipes regardless of location.

If you need to run it on all field names, then you can try using a Foreach processor on the _source and it'll run the Gsub against all the top-level fields. If you need to to apply it to subfields as well, then you get into recursion and it can get complicated fast, depending on number of levels and uniformity.
You can always create a custom pipeline. With the script processors you can do a whole lot of things, but when you start to get into this level of complexity, it might be worth taking a step back and re-examining the problem starting with where the content is being indexed from and how...

Spend some time reading the helpdocs, and think outside the box. You can do almost anything. Some things scale and some don't.

Also, generally speaking, try to give more specifics on your use case when asking for help. Include examples.

Hope that helps.

5

Hi Scott, thank you for the reply!

i need to replace values only in field names and top level fields. I will give Gsub a try.

[Edit]:

solved the issue with ingest pipeline.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.