# Logstash Enrich and translate plugin use **URL:** \ **Category:** Logstash\ **Created:** [May 9, 2023, 9:31am UTC](https://discuss.elastic.co/t/logstash-enrich-and-translate-plugin-use/332897 "2023-05-09T09:31:03Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![gbandasha](https://avatars.discourse-cdn.com/v4/letter/g/c0e974/32.png) [@gbandasha](https://discuss.elastic.co/u/gbandasha)\ **Post date:** [May 9, 2023, 9:31am UTC](https://discuss.elastic.co/t/logstash-enrich-and-translate-plugin-use/332897/1 "2023-05-09T09:31:03Z")
Hello Team, I am trying to enrich the data before it makes its way too elastic, I have tried the below methods but both are currently not working 1. Using the elasticsearch plugin in filter ```auto input { kafka { bootstrap_servers => "xxx.xx.xx.xxx:9092" topics => ["topicname"] } } filter { elasticsearch { hosts => ["xxx.xx.xx.xxx:9200"] index => "lookup" query => '{ "query": { "match": { "id": "%{id}" } } }' fields => { "desc" => "desc" } user => elastic password => " ******" ca_file => "/path/to/elasticsearch-ca.pem" } json { source => "message" remove_field => ["message", "@version"] } mutate{ lowercase => ["name"] } } output { elasticsearch { hosts => ["xxx.xx.xx.xxx:9200"] index => "test" document_id => "%{id}" user => elastic password => ******* ssl => true cacert => '/path/to/elasticsearch-ca.pem' ssl_certificate_verification => false } } ``` The issue that I ran into is that **ssl\_certificate\_verification** is not allowed as a parameter on the Elasticsearch filter hence it is not connecting to Elasticsearch. I tried generating the cert using the following link so that I can have a secure SSL connection but that also doesn't work > **[Configure SSL/TLS for the Logstash output | Fleet and Elastic Agent Guide...](https://www.elastic.co/guide/en/fleet/current/secure-logstash-connections.html)** 2)After which I tried the translate filter to achieve the same, I get no error but the lookup using the dictionary is not happening ```auto input { kafka { bootstrap_servers => "xxx.xx.xx.xxx:9092" topics => ["topicname"] } } filter { translate { source => ["code"] target => ["desc"] dictionary => { "1" => "One" "2" => "two" "3" => "three" "4" => "four" } override => true fallback => "Fallback" } json { source => "message" remove_field => ["message", "@version"] } mutate{ lowercase => ["name"] } } output { stdout { codec => rubydebug } elasticsearch { hosts => ["xxx.xx.xx.xxx:9200"] index => "test" document_id => "%{id}" user => elastic password => " *********" ssl => true cacert => '/path/to/elasticsearch-ca.pem' ssl_certificate_verification => false } } ``` The value of id was INTEGER but the dictionary source field should be a string so I even converted and tried no error but the desc field is blank. Finally I also setup the enrich index and configuration on elastic but when I send data through Logstash I get DLQ issues. Regards , Guru ---
**Author:** ![gbandasha](https://avatars.discourse-cdn.com/v4/letter/g/c0e974/32.png) [@gbandasha](https://discuss.elastic.co/u/gbandasha)\ **Post date:** [May 11, 2023, 8:36am UTC](https://discuss.elastic.co/t/logstash-enrich-and-translate-plugin-use/332897/2 "2023-05-11T08:36:26Z")
I am still not able to get it working. I was able to successfully setup the elastic enrich pipeline, index template, and add records from the Dev Tools PUT cmd but the through logstash is not working. ---
**Author:** ![gbandasha](https://avatars.discourse-cdn.com/v4/letter/g/c0e974/32.png) [@gbandasha](https://discuss.elastic.co/u/gbandasha)\ **Post date:** [May 24, 2023, 8:11am UTC](https://discuss.elastic.co/t/logstash-enrich-and-translate-plugin-use/332897/3 "2023-05-24T08:11:37Z")
Any update on this will be helpful to debug the issue with Enrich pipeline. ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [May 24, 2023, 1:39pm UTC](https://discuss.elastic.co/t/logstash-enrich-and-translate-plugin-use/332897/4 "2023-05-24T13:39:52Z")
Can you provide more context? It is no clear what you are trying to do now since you mentioned translate filter and elasticsearch filter in Logstash and Enrich pipeline in Elasticsearch which are two completely different things. I would say that if you are using Logstash the easiest way to enrich data is using the translate filter. You didn't share any sample message of how your data looks like, nor any log error you are getting, so it is not possible to know why the translate filter didn't worked. Also, in the logstash configuration you shared you had your translate filter **before** your json filter, it needs to come after you parse your message or the source field will not exist and the translate filter will not work. ---
**Author:** ![gbandasha](https://avatars.discourse-cdn.com/v4/letter/g/c0e974/32.png) [@gbandasha](https://discuss.elastic.co/u/gbandasha)\ **Post date:** [May 28, 2023, 8:10am UTC](https://discuss.elastic.co/t/logstash-enrich-and-translate-plugin-use/332897/5 "2023-05-28T08:10:05Z")
Hello @leandrojmp , Thanks a lot for pointing that out I was able to get the translate working and instead of using dictionary I have used the dictionary path variable but I noticed that every time we update the values on the CSV file we need to restart logstash is that right ? Also I am using multiple translate in the filter for performing multiple lookups. ```auto input { kafka { bootstrap_servers => "xxx.xx.xx.xxx:9092" topics => ["topicname"] } } filter { json { source => "message" remove_field => ["message", "@version"] } mutate{ lowercase => ["name"] } translate { source => ["code"] target => ["desc"] dictionary => { "1" => "One" "2" => "two" "3" => "three" "4" => "four" } override => true fallback => "Fallback" } translate { source => ["code2"] target => ["desc2"] dictionary => { "1" => "Black" "2" => "Green" "3" => "Yellow" "4" => "Red" } override => true fallback => "Fallback" } } output { stdout { codec => rubydebug } elasticsearch { hosts => ["xxx.xx.xx.xxx:9200"] index => "test" document_id => "%{id}" user => elastic password => " *********" ssl => true cacert => '/path/to/elasticsearch-ca.pem' ssl_certificate_verification => false } } ``` Regards, Guru ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [May 28, 2023, 11:49am UTC](https://discuss.elastic.co/t/logstash-enrich-and-translate-plugin-use/332897/6 "2023-05-28T11:49:30Z")
> [@gbandasha](#): > > I noticed that every time we update the values on the CSV file we need to restart logstash is that right ? No, the dictionary is refreshed in memory, per default logstash will check the dictionary for changes and refresh it every 5 minutes, but you can decrease the [refresh interval](https://www.elastic.co/guide/en/logstash/current/plugins-filters-translate.html#plugins-filters-translate-refresh_interval). ---
**Author:** ![gbandasha](https://avatars.discourse-cdn.com/v4/letter/g/c0e974/32.png) [@gbandasha](https://discuss.elastic.co/u/gbandasha)\ **Post date:** [May 31, 2023, 4:08pm UTC](https://discuss.elastic.co/t/logstash-enrich-and-translate-plugin-use/332897/7 "2023-05-31T16:08:27Z")
Thanks @leandrojmp It does sync after 5 min. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 28, 2023, 4:08pm UTC](https://discuss.elastic.co/t/logstash-enrich-and-translate-plugin-use/332897/8 "2023-06-28T16:08:47Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.