Using reindex to clean up a series of indexes (6.8.12)

Elastic Stack Elasticsearch
1

Background:

I am attempting to get a series of indexes into a consistent form -- the indexes span two years and over that time I have learnt a lot and there have been incremental improvements in the data quality.

The index now has a mapping specified and I don't want anything in the reindexed data that is not in the mapping.

There are also fields that have changed their names and I want to remap these and lastly there are IP addresses that now get geoip data added and I would like to do this for older entries too.

I see there is script option for reindex but the documentation is scant.

Specific questions:
1/ is there a way to tell ES not to put any fields into a doc that is not in the index mapping?

2/ can I use "painless" scripts to clean up the data during the reindexing process? --if so an example of renaming a field, or reformatting a field would be appreciated.

3/ deleting specific fields from the index (not necessary if I can do this in the mapping)

2
  1. Use dynamic: false - https://www.elastic.co/guide/en/elasticsearch/reference/current/dynamic.html
  2. Yeah you can, that's not my best area so hopefully someone else will jump in.
  3. See 1.
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.