# Bug with logstash-filter-elasticsearch

**URL:** <https://discuss.elastic.co/t/bug-with-logstash-filter-elasticsearch/50161>\
**Category:** Logstash\
**Created:** [May 16, 2016, 10:20pm UTC](https://discuss.elastic.co/t/bug-with-logstash-filter-elasticsearch/50161 "2016-05-16T22:20:29Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Andrew\_Shved](https://avatars.discourse-cdn.com/v4/letter/a/4af34b/32.png) [@Andrew\_Shved](https://discuss.elastic.co/u/Andrew_Shved)\
**Post date:** [May 16, 2016, 10:20pm UTC](https://discuss.elastic.co/t/bug-with-logstash-filter-elasticsearch/50161/1 "2016-05-16T22:20:29Z")

</div>

I have tried example straight from documentation and it seems to fail as elasticsearch expects " around field names.  
elasticsearch {  
hosts =\> ["localhost"]  
query =\> "statuscode:200"  
fields =\> ["commonid", "commonid"]  
}  
basically implementing a lookup table

logstash has this error  
:message=\>"Failed to query elasticsearch for previous event", :query=\>"statuscode:200"  
while elasticsearch has this  
RemoteTransportException[[Jude the Entropic Man][172.17.0.2:9300][indices:data/read/search[phase/query]]]; nested: SearchParseException[failed to parse search source [{"size":1,"query":{"query\_string":{"query":"statuscode:200","lowercase\_expanded\_terms":true,"analyze\_wildcard":false}},"sort":[{"@timestamp":{"order":"desc"}}]}]]; nested: SearchParseException[No mapping found for [@timestamp] in order to sort on];

when i try to single qute the expression and add double quotes around the fields it still fails ☹

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [May 17, 2016, 6:03am UTC](https://discuss.elastic.co/t/bug-with-logstash-filter-elasticsearch/50161/2 "2016-05-17T06:03:06Z")

</div>

> [@Andrew\_Shved](#):
>
> No mapping found for [@timestamp] in order to sort on

Has nothing to do with double quotes.

> [@Andrew\_Shved](#):
>
> when i try to single qute the expression and add double quotes around the fields it still fails

Can you post an example? Make sure you format your post correctly with the code formatting button.

---

<div class="post-metadata">

**Author:** ![Andrew\_Shved](https://avatars.discourse-cdn.com/v4/letter/a/4af34b/32.png) [@Andrew\_Shved](https://discuss.elastic.co/u/Andrew_Shved)\
**Post date:** [May 17, 2016, 4:12pm UTC](https://discuss.elastic.co/t/bug-with-logstash-filter-elasticsearch/50161/3 "2016-05-17T16:12:57Z")

</div>

thanks so the issue is .kibana index which does not have @timestamp. After I added the querry works without fields parameter but when I added to get the output back into one of the fields of the new event

if [type] == "cache" {  
elasticsearch {  
hosts =\> ["[http://localhost:9200](http://localhost:9200)"]  
query =\> "type:cache"  
fields =\> ["sessionid", "message"]  
}  
}

I get the following in the logstash log. Not sure what I am doing wrong as I simplified to makes sure that sessionid exists on the retrieval an message field exists in the new event...

{:timestamp=\>"2016-05-17T16:02:05.382000+0000", :message=\>"Failed to query elasticsearch for previous event", :query=\>"type:cache", :event=\>#\<LogStash::Event:0x2e30991a @metadata={}, @accessors=#\<LogStash::Util::Accessors:0x789391a6 @store={"@version"=\>"1", "@timestamp"=\>"2016-05-17T16:01:59.613Z", "type"=\>"cache", "beat"=\>{"hostname"=\>"2794669d9139", "name"=\>"2794669d9139"}, "input\_type"=\>"log", "count"=\>1, "fields"=\>nil, .... "sessionid"=\>"fe666e8e-a44e-425b-9dbb-6bd0815519b1", "client\_sessionid"=\>"fe666e8e-a44e-425b-9dbb-6bd0815519a1"}, @lut={"type"=\>[{"@version"=\>"1", "@timestamp"=\>"2016-05-17T16:01:59.613Z", "type"=\>"cache", "beat"=\>{"hostname"=\>"2794669d9139", "name"=\>"2794669d9139"}, "input\_type"=\>"log", "count"=\>1, "fields"=\>nil, "source"=\>"/dockershare/serviceB.log", "offset"=\>36295, "host"=\>"2794669d9139", "tags"=\>["beats\_input\_codec\_plain\_applied"], "timestamp"=\>"2016-05-03T11:53:15.761Z", "sessionid"=\>"fe666e8e-a44e-425b-9dbb-6bd0815519b1", "client\_sessionid"=\>"fe666e8e-a44e-425b-9dbb-6bd0815519a1"}, "type"], "logregion"=\>[{"@version"=\>"1", "@timestamp"=\>"2016-05-17T16:01:59.613Z", "type"=\>"cache", "beat"=\>{"hostname"=\>"2794669d9139", "name"=\>"2794669d9139"}, "input\_type"=\>"log", "count"=\>1, "fields"=\>nil, "...., "timestamp"=\>"2016-05-03T11:53:15.761Z", "sessionid"=\>"fe666e8e-a44e-425b-9dbb-6bd0815519b1", "client\_sessionid"=\>"fe666e8e-a44e-425b-9dbb-6bd0815519a1"}, "logregion"], "loglevel"=\>[{"@version"=\>"1", "@timestamp"=\>"2016-05-17T16:01:59.613Z", "type"=\>"cache", "beat"=\>{"hostname"=\>"2794669d9139", "name"=\>"2794669d9139"}, "input\_type"=\>"log", "count"=\>1, "fields"=\>nil, "source"=\>"/dockershare/serviceB.log", "offset"=\>36295, "host"=\>"2794669d9139", "tags"=\>["beats\_input\_codec\_plain\_applied"], "timestamp"=\>"2016-05-03T11:53:15.761Z", "sessionid"=\>"fe666e8e-a44e-425b-9dbb-6bd0815519b1", "client\_sessionid"=\>"fe666e8e-a44e-425b-9dbb-6bd0815519a1"}, "loglevel"], "client\_requestid"=\>[{"@version"=\>"1", "@timestamp"=\>"2016-05-17T16:01:59.613Z", "type"=\>"cache", "beat"=\>{"hostname"=\>"2794669d9139", "name"=\>"2794669d9139"}, "input\_type"=\>"log", "count"=\>1, "fields"=\>nil, "source"=\>"/dockershare/serviceB.log", "offset"=\>36295, "host"=\>"2794669d9139", "tags"=\>["beats\_input\_codec\_plain\_applied"], "timestamp"=\>"2016-05-03T11:53:15.761Z", "sessionid"=\>"fe666e8e-a44e-425b-9dbb-6bd0815519b1", "client\_sessionid"=\>"fe666e8e-a44e-425b-9dbb-6bd0815519a1"}, "client\_requestid"], "message"=\>[{"@version"=\>"1", "@timestamp"=\>"2016-05-17T16:01:59.613Z", "type"=\>"cache", "beat"=\>{"hostname"=\>"2794669d9139", "name"=\>"2794669d9139"}, "input\_type"=\>"log", "count"=\>1, "fields"=\>nil, "source"=\>"/dockershare/serviceB.log", "offset"=\>36295, "host"=\>"2794669d9139", "tags"=\>["beats\_input\_codec\_plain\_applied"], "timestamp"=\>"2016-05-03T11:53:15.761Z", "sessionid"=\>"fe666e8e-a44e-425b-9dbb-6bd0815519b1", "client\_sessionid"=\>"fe666e8e-a44e-425b-9dbb-6bd0815519a1"}, "message"], "[type]"=\>[{"@version"=\>"1", "@timestamp"=\>"2016-05-17T16:01:59.613Z", "type"=\>"cache", "beat"=\>{"hostname"=\>"2794669d9139", "name"=\>"2794669d9139"}, "input\_type"=\>"log", "count"=\>1, "fields"=\>nil, "source"=\>"/dockershare/serviceB.log", "offset"=\>36295, "host"=\>"2794669d9139", "tags"=\>["beats\_input\_codec\_plain\_applied"], "timestamp"=\>"2016-05-03T11:53:15.761Z", "sessionid"=\>"fe666e8e-a44e-425b-9dbb-6bd0815519b1", "client\_sessionid"=\>"fe666e8e-a44e-425b-9dbb-6bd0815519a1"}, "type"]}\>, @data={"@version"=\>"1", "@timestamp"=\>"2016-05-17T16:01:59.613Z", "type"=\>"cache", "beat"=\>{"hostname"=\>"2794669d9139", "name"=\>"2794669d9139"}, "input\_type"=\>"log", "count"=\>1, "fields"=\>nil, "source"=\>"/dockershare/serviceB.log", "offset"=\>36295, "host"=\>"2794669d9139", "tags"=\>["beats\_input\_codec\_plain\_applied"], "timestamp"=\>"2016-05-03T11:53:15.761Z", "sessionid"=\>"fe666e8e-a44e-425b-9dbb-6bd0815519b1", "client\_sessionid"=\>"fe666e8e-a44e-425b-9dbb-6bd0815519a1"}, @metadata\_accessors=#\<LogStash::Util::Accessors:0x2e9ad16d @store={}, @lut={}\>, @cancelled=false\>, :error=\>#\<NoMethodError: undefined method `start\_with?' for nil:NilClass\>, :level=\>:warn}  
~  
~  
~

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [May 17, 2016, 4:53pm UTC](https://discuss.elastic.co/t/bug-with-logstash-filter-elasticsearch/50161/4 "2016-05-17T16:53:10Z")

</div>

Check your ES log, there may be more there.

---

<div class="post-metadata">

**Author:** ![Andrew\_Shved](https://avatars.discourse-cdn.com/v4/letter/a/4af34b/32.png) [@Andrew\_Shved](https://discuss.elastic.co/u/Andrew_Shved)\
**Post date:** [May 17, 2016, 5:10pm UTC](https://discuss.elastic.co/t/bug-with-logstash-filter-elasticsearch/50161/5 "2016-05-17T17:10:36Z")

</div>

did no errors. I am questioning part when I map fields and it returns fields =\> nil. I tried running the same query on elasticsearch manually and get expected results. wonder if fields parameter is a real issue.

I am just implementing "cache" for some type of log events. my PoC depends on looking up sessionids in es cache for some type of events and copying the value to the new event.

---

<div class="post-metadata">

**Author:** ![Andrew\_Shved](https://avatars.discourse-cdn.com/v4/letter/a/4af34b/32.png) [@Andrew\_Shved](https://discuss.elastic.co/u/Andrew_Shved)\
**Post date:** [May 17, 2016, 9:10pm UTC](https://discuss.elastic.co/t/bug-with-logstash-filter-elasticsearch/50161/6 "2016-05-17T21:10:28Z")

</div>

Ok so it works with logstash 2.2 but not 2.3. Thus I have to downgrade. Not sure if this community plugin will be actively supported ☹ to fix this issue

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:57am UTC](https://discuss.elastic.co/t/bug-with-logstash-filter-elasticsearch/50161/7 "2017-07-06T04:57:20Z")

</div>


