# How to use translate plugin with ES index for lookup data **URL:** \ **Category:** Logstash\ **Created:** [July 23, 2019, 6:27am UTC](https://discuss.elastic.co/t/how-to-use-translate-plugin-with-es-index-for-lookup-data/191743 "2019-07-23T06:27:05Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![msk\_76](https://avatars.discourse-cdn.com/v4/letter/m/dbc845/32.png) [@msk\_76](https://discuss.elastic.co/u/msk_76)\ **Post date:** [July 23, 2019, 6:27am UTC](https://discuss.elastic.co/t/how-to-use-translate-plugin-with-es-index-for-lookup-data/191743/1 "2019-07-23T06:27:05Z")
I have read about translate filter functionality in logstash which could be used for lookup event data with that of a file( csv, json) which we can define in destination\_path yml file. Can I use an elastisearch index ( containing the key-value lookup data) to be used instead of csv/json file? This is due to the reason that my event data is in one index and my lookup data is in another index and I want to merge them based on key value. The lookup data is huge to be kept in a csv/json file. if it's not possible with translate, is there any other way or other filter? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [July 23, 2019, 7:09am UTC](https://discuss.elastic.co/t/how-to-use-translate-plugin-with-es-index-for-lookup-data/191743/2 "2019-07-23T07:09:57Z")
You will need to use the Elasticsearch filter plugin. Be aware that this can be slow though as it requires a network round trip per event. ---
**Author:** ![msk\_76](https://avatars.discourse-cdn.com/v4/letter/m/dbc845/32.png) [@msk\_76](https://discuss.elastic.co/u/msk_76)\ **Post date:** [July 24, 2019, 5:48am UTC](https://discuss.elastic.co/t/how-to-use-translate-plugin-with-es-index-for-lookup-data/191743/3 "2019-07-24T05:48:43Z")
I read about the elasticsearch plugin filter but it seems that it can refer to old events in same index. My requirement is to do lookup based on timestamp, cluster, index1 timestamp|cluster|jobid|usage index2 timestamp|cluster|jobid|hostname index3 hostname|hostgroup Logic of lookup is if max(timestamp) of index2 \> max(timestamp) of index2 then left outer join index1 with index2 with index3 on index1.timestamp = index2.timestamp and index1.cluster = index2.cluster and index1.jobid = index2.jobid index2.hostname = index3.hostname The resultant output would be index4 timestamp|cluster|jobid|usage|hostname|hostgroup ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [July 24, 2019, 7:10am UTC](https://discuss.elastic.co/t/how-to-use-translate-plugin-with-es-index-for-lookup-data/191743/4 "2019-07-24T07:10:41Z")
Elasticsearch does not support joins so you may need to express this as multiple separate Elasticsearch lookups. I not sure I follow the logic so may be wrong though. ---
**Author:** ![msk\_76](https://avatars.discourse-cdn.com/v4/letter/m/dbc845/32.png) [@msk\_76](https://discuss.elastic.co/u/msk_76)\ **Post date:** [July 24, 2019, 7:41am UTC](https://discuss.elastic.co/t/how-to-use-translate-plugin-with-es-index-for-lookup-data/191743/5 "2019-07-24T07:41:19Z")
Ok I can do it with multiple ES lookups connected sequentially but how to implement this condition if [max(timestamp)] of index2 \> [max(timestamp)] of index1 ? timestamp is the field present in both index1 & index2. If the above condition is satisfied then only parse the logstash input. Please suggest. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [July 24, 2019, 8:02am UTC](https://discuss.elastic.co/t/how-to-use-translate-plugin-with-es-index-for-lookup-data/191743/6 "2019-07-24T08:02:23Z")
I do not think you can do that so may need to restructure the indexes. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 21, 2019, 8:02am UTC](https://discuss.elastic.co/t/how-to-use-translate-plugin-with-es-index-for-lookup-data/191743/7 "2019-08-21T08:02:24Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.