Add description to my logstas index based on another csv field

Elastic Stack Elasticsearch
1

I have my index created using logstah config file. Reading from log file following information:

id, iduser,datetimInit, dateTimeends
0001 210 2023-02-03 04:45:16.78 2023-02-03 04:46:16.78
0002 1003 2023-02-03 08:45:16.78 2023-02-03 08:46:16.78
0003 210 2023-02-04 04:45:16.78 2023-02-04 04:46:16.78

This is my config file:

filter {	
	grok {
		match => { 
			"message" => "%{NUMBER:id} %{NUMBER:idUser} %{TIMESTAMP_ISO8601:dateTimeInit} %{TIMESTAMP_ISO8601:dateTimeEnds}" 
		}
	}
}

On another csv file I have de full name if all users like that:

iduser,fullusername
210,Rosa Leon
200,Ana Quintero
1003,Marta Torres

I need to add user name field (fullname) from another csv file with this structure:

I do not hnow how to get user full name to add to my index from csv file and match by iduser. I nedd to get this full information into mi index using logstash file configuration:

0001 210 2023-02-03 04:45:16.78 2023-02-03 04:46:16.78 Rosa Leon
0002 1003 2023-02-03 08:45:16.78 2023-02-03 08:46:16.78 Marta Torres
0003 210 2023-02-04 04:45:16.78 2023-02-04 04:46:16.78 Rosa Leon

Matched by idUser

2

You can do that with the translate filter.

3

Thank you!
After reading the documentation, I have included:

      filter {
          csv {
          columns => ["iduser","fullusername"]
          separator => ","
          }
	translate {
		dictionary_path => "<path>/userdescripcion.csv"
		source => "fullusername"   #second column of my csv file userdescripcion.csv
		target => "iduser" #identifier
    }
}

But nothing happends, Any suggestion?

4

Why you added this csv filter? You do not need this, you are already parsing your message with the grok filter, right?

filter {	
	grok {
		match => { 
			"message" => "%{NUMBER:id} %{NUMBER:idUser} %{TIMESTAMP_ISO8601:dateTimeInit} %{TIMESTAMP_ISO8601:dateTimeEnds}" 
		}
	}
}

This creates the idUser field that you will use in the translate filter as the source.

Your translate filter needs to be something like this:

	translate {
		dictionary_path => "<path>/userdescripcion.csv"
		source => "idUser"
		target => "fullusername"
    }

And your userdescripcion.csv should look like this:

210,Rosa Leon
200,Ana Quintero
1003,Marta Torres
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.