Reindex Elasticsearch using new mapping

Elastic Stack Logstash
1

Hi ,

I have 2 new fields added to the index and i want to renidex the old indices to include these two fields. I have a mapping csv file where i have an existing field and corresponding value for two new fields. I want to use this csv and re index the old indices?
can anyone help?

2

I'm not able to follow what you're trying to do.

Mapping has meaning to Elasticsearch, so this is harder to parse.

This is confusing as well. A reindex operation doesn't necessarily need anything new to work. If I had to guess at what you're describing, it's that you want to use the elasticsearch input plugin to read your existing index, and then use the values from this csv to somehow convert fields in the old data into two new fields for the target indices? Is this correct?

If so, you should read up on the translate filter plugin, which should allow you to use the csv to accomplish this.

3

whatever the you guessed was correct and thanks for suggesting logstash translate plugin. I was using ruby filter plugin and written a small code snippet to parse the csv . Translate plugin will do same without any effort.

I have one more question , Can we add two fields from csv or yaml file in a single translate filter block? when i checked i could see the destination is a string and always allows single field.
So i think in my case i need to write two filer blocks?

4

Yes, sorry. Translate only allows one match right now.

5

The match could however be a JSON string that you then can decode using a JSON filter.

2 Likes
6

I have got another option. Storing the multiple fields from translate filter to csv and then use the csv filter to parse it.
below link has the details.

7

thanks , same i did it with csv filter .

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.