# Logstash-filter-elasticsearch plugin fails to look up in amazon es **URL:** \ **Category:** Elasticsearch\ **Created:** [January 21, 2018, 12:07am UTC](https://discuss.elastic.co/t/logstash-filter-elasticsearch-plugin-fails-to-look-up-in-amazon-es/116362 "2018-01-21T00:07:24Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![srama](https://avatars.discourse-cdn.com/v4/letter/s/c67d28/32.png) [@srama](https://discuss.elastic.co/u/srama)\ **Post date:** [January 21, 2018, 12:07am UTC](https://discuss.elastic.co/t/logstash-filter-elasticsearch-plugin-fails-to-look-up-in-amazon-es/116362/1 "2018-01-21T00:07:24Z")
Hi, New to ELK. I am trying to use logstash-filter-elasticsearch plugin for some look up and computation. It works fine with on prem 5.6.3 version. The same config fails while connection to Amazon ES 6.0. Here is my config sample. filter { if "ABC.process" in [service\_name] { elasticsearch { hosts =\> ["[someprivatecloud.amazonaws.com:443](http://someprivatecloud.amazonaws.com:443)"] ssl =\> true index =\> "testindex" query =\> 'service\_name:"XYZ.process" AND transnumber:%{[transnumber]}' fields =\> {"logtimestamp" =\> "startdate" } } ruby { init =\> "require 'time'" code =\> "duration = ((Time.parse(event.get('logtimestamp')[0]).to\_f \* 1000) - (Time.parse(event.get('startdate')[0]).to\_f \* 1000)) rescue nil; event.set('endtime', duration);" } } } ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [January 21, 2018, 12:30am UTC](https://discuss.elastic.co/t/logstash-filter-elasticsearch-plugin-fails-to-look-up-in-amazon-es/116362/2 "2018-01-21T00:30:28Z")
Can you elaborate on the failure? ---
**Author:** ![srama](https://avatars.discourse-cdn.com/v4/letter/s/c67d28/32.png) [@srama](https://discuss.elastic.co/u/srama)\ **Post date:** [January 21, 2018, 1:17am UTC](https://discuss.elastic.co/t/logstash-filter-elasticsearch-plugin-fails-to-look-up-in-amazon-es/116362/3 "2018-01-21T01:17:00Z")
All I see is a tag with "\_elasticsearch\_lookup\_failure" I do not know how to enable debug/trace within "filter". I can provide any trace if you can direct the instructions to capture. ( Thats would be amazing help ) Thank you in advance. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [January 21, 2018, 2:31am UTC](https://discuss.elastic.co/t/logstash-filter-elasticsearch-plugin-fails-to-look-up-in-amazon-es/116362/4 "2018-01-21T02:31:57Z")
That implies nothing matched your query, so I would check that there is indeed something that matches. ---
**Author:** ![srama](https://avatars.discourse-cdn.com/v4/letter/s/c67d28/32.png) [@srama](https://discuss.elastic.co/u/srama)\ **Post date:** [January 21, 2018, 7:16am UTC](https://discuss.elastic.co/t/logstash-filter-elasticsearch-plugin-fails-to-look-up-in-amazon-es/116362/5 "2018-01-21T07:16:35Z")
I am certain that the data exists as the same is working fine with 5.6 onprem ES instance. Is there anyway else I could troubleshoot? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [January 21, 2018, 11:35am UTC](https://discuss.elastic.co/t/logstash-filter-elasticsearch-plugin-fails-to-look-up-in-amazon-es/116362/6 "2018-01-21T11:35:58Z")
In order to authenticate with AWS Elasticsearch service, a special Elasticsearch output plugin provided by Amazon (`logstash-output-amazon_es`) is required. I suspect this would also be a problem for the standard Elasticsearch filter plugin, so you may want to ask AWS support if they have a version of the Elasticsearch filter plugin. These standard Logstash plugins should however work with [Elastic Cloud](https://www.elastic.co/cloud) as it has a different authentication mechanism, so that might also be an option. ---
**Author:** ![srama](https://avatars.discourse-cdn.com/v4/letter/s/c67d28/32.png) [@srama](https://discuss.elastic.co/u/srama)\ **Post date:** [January 21, 2018, 4:34pm UTC](https://discuss.elastic.co/t/logstash-filter-elasticsearch-plugin-fails-to-look-up-in-amazon-es/116362/7 "2018-01-21T16:34:23Z")
Finally, I am able to lookup in amazon es instance. All I had to do was change these two lines within filter's elasticsearch hosts =\> ["[someprivatecloud.amazonaws.com:443](http://someprivatecloud.amazonaws.com:443)"] ssl =\> true to hosts =\> ["[https://someprivatecloud.amazonaws.com:443](https://someprivatecloud.amazonaws.com:443)"] removed this line -- ssl =\> true Thank you all !! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 18, 2018, 4:34pm UTC](https://discuss.elastic.co/t/logstash-filter-elasticsearch-plugin-fails-to-look-up-in-amazon-es/116362/8 "2018-02-18T16:34:30Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.