Reingenst CSV to change field names

Elastic Stack Kibana
1

Hello, I'm new to ELK stack.

I have a lot of CSV files which share some common columns.
But all have different names for their columns.

Example:
File 1: Name, Type, Code
File 2: TP, NM, CD
File 3: Co,Ty,Na

My code needs to reference them with the same name, so I need to normalize the names on ingest (I think).

So I changed the column names on ingest to PRE_NAME, PRE_CODE, PRE_TYPE

Now I made a mistake, forgot the underscore and ingested one as:
PRENAME, PRECODE, PRETYPE

Is there a simple way to change that and reingest or do I always have to completely delete index and data view and recreate the whole thing again?

My examples are made easy but I have around 50 columns so recreating everytime is painfull.

greetings, Martin

2

Hi Martin,

You can use rename processor in ingest pipeline and re-index api to achieve above.

  1. Create a Ingest pipeline with rename processor and enter field names.
  2. Reindex the data into same indices by applying pipeline,
1 Like
3

Hello, thank you for pointing me to the right methods.
I tried to use the reindex.
But if I try to use my index as source and dest, I get an error.
How can I reindex the CSV again?
I mean, how do I add it to the source of the _reindex ?
Sorry if I missed something in the docs but I cans find how to set a CSV as source.

I used Kibana Machine Learning File import to do the first import

Ah, maybe I have to import the file in a index without changing anything first like myFile_native and then do a reindex into a new one like myFile_foruse with my changes and use the myFile_foruse in the application right?

4

lets say you have your data in index1 currently , Just reindex by keeping index1 as source and index2(a new index) as a dest by applying pipeline.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.