Lookup Values

Elastic Stack Logstash
1

I have CDR file contain 100K rows.
Some of the columns in that CDR file contain ID columns.
I need to do a lookup and fetch the description of those ID columns.
For columns that has small set of ID`s descriptions , i haved added the following entry in the config file

    translate {
        field => "P10"
        destination => "CALL_TYPE_DESC"
        dictionary => {
			"0" => "Unknown"
			"1" => "Local"
                                               ....
                                                                    "100" => "End"
                                             }
                     }

What are my options when i need to fetch ID description from files/db tables that contain 20,000 rows or more

2

Hi Experts,
As i wrote , I have CDR file contain 100K rows.
I am using csv filter and successfully uploaded the rows using logstatsh.
Some of the columns in that CDR file contain ID columns.
I need to do a lookup and fetch the description of those ID columns from ElasticSearch table.
So actually , for each and every line in the csv file , i need to do a lookup that fetch the description of the ID column.
For example the csv file contain CountryId coulmn . so for CountryId = 1 , i need to do a lookup that fetch the description of : USA
Please advise how to lookup at remote elasticsearch table while filtering a csv file
Thanks
Yoav

3

You can use the translate filter with a file as input or the jdbc_static filter if you have the data in a rdbms.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.