# Unable to retreive data from index using elasticsearch input plugin in logstash

**URL:** <https://discuss.elastic.co/t/unable-to-retreive-data-from-index-using-elasticsearch-input-plugin-in-logstash/181375>\
**Category:** Logstash\
**Created:** [May 16, 2019, 10:52am UTC](https://discuss.elastic.co/t/unable-to-retreive-data-from-index-using-elasticsearch-input-plugin-in-logstash/181375 "2019-05-16T10:52:50Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Vishnu\_mk](https://avatars.discourse-cdn.com/v4/letter/v/71c47a/32.png) [@Vishnu\_mk](https://discuss.elastic.co/u/Vishnu_mk)\
**Post date:** [May 16, 2019, 10:52am UTC](https://discuss.elastic.co/t/unable-to-retreive-data-from-index-using-elasticsearch-input-plugin-in-logstash/181375/1 "2019-05-16T10:52:50Z")

</div>

```
    input {
  elasticsearch {
    hosts => ["172.25.8.45:9200"]
    index => ["heartbeat-*"]
   # query => '{ "query": { "match_all": {} } }'
   # query => '{ "query": { "bool": {"must" : { "term" : { "monitor.status" : "down" } } } } }'
   # query => '{ "query": { "query_string": { "query": "*" } } }'
   # size => 500
   # scroll => "5m"
    #docinfo => true
  }
}
output {
   # if "_grokparsefailure" in [tags] {
  stdout { codec => "rubydebug" }
 # }
    elasticsearch {
    hosts => ["172.25.8.45:9200"]
    manage_template => false
    index => "tp"
   # document_type => "new-type"
     }
}

```

This is my logstash configuration

Im getting following error:

```
[2019-05-16T16:15:59,334][ERROR][logstash.pipeline] A plugin had an unrecoverable error. Will restart this plugin.
  Pipeline_id:main
  Plugin: <LogStash::Inputs::Elasticsearch index=>"heartbeat-*", hosts=>["172.25.8.45:9200"], id=>"49529869826fbe6db5a4ea023f4022b6ebb73e17e4ff1830d2ffb45b9b292152", enable_metric=>true, codec=><LogStash::Codecs::JSON id=>"json_ef14a6ee-dfb5-4070-9a52-faa253c049c8", enable_metric=>true, charset=>"UTF-8">, query=>"{ \"sort\": [\"_doc\"] }", size=>1000, scroll=>"1m", docinfo=>false, docinfo_target=>"@metadata", docinfo_fields=>["_index", "_type", "_id"], ssl=>false>
  Error: [411] <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<HTML><HEAD><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<TITLE>ERROR: The requested URL could not be retrieved</TITLE>
<STYLE type="text/css"><!--BODY{background-color:#ffffff;font-family:verdana,sans-serif}PRE{font-family:sans-serif}--></STYLE>
</HEAD><BODY>
<H1>ERROR</H1>
<H2>The requested URL could not be retrieved</H2>
<HR noshade size="1px">
<P>
While trying to process the request:
<PRE>
GET /heartbeat-%2A/_search?scroll=1m&amp;size=1000 HTTP/1.1
Content-Type: application/json
User-Agent: Faraday v0.9.2
Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3
Accept: */*
Connection: close
Host: 172.25.8.45:9200
Content-Length: 22

</PRE>
<P>
The following error was encountered:
<UL>
<LI>
<STRONG>
Invalid Request
</STRONG>
</UL>

<P>
Some aspect of the HTTP Request is invalid. Possible problems:
<UL>
<LI>Missing or unknown request method
<LI>Missing URL
<LI>Missing HTTP Identifier (HTTP/1.0)
<LI>Request is too large
<LI>Content-Length missing for POST or PUT requests
<LI>Illegal character in hostname; underscores are not allowed
</UL>
<P>Your cache administrator is <A HREF="mailto:root">root</A>. 

<BR clear="all">
<HR noshade size="1px">
<ADDRESS>
Generated Thu, 16 May 2019 09:00:09 GMT by TGPROXY-Backup (squid/2.6.STABLE6)
</ADDRESS>
</BODY></HTML>

  Exception: Elasticsearch::Transport::Transport::Errors::LengthRequired
  Stack: /home/abc/elk/logstash-6.5.3/vendor/bundle/jruby/2.3.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/transport/base.rb:202:in `__raise_transport_error'
/home/abc/elk/logstash-6.5.3/vendor/bundle/jruby/2.3.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/transport/base.rb:319:in `perform_request'
/home/abc/elk/logstash-6.5.3/vendor/bundle/jruby/2.3.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/transport/http/faraday.rb:20:in `perform_request'
/home/abc/elk/logstash-6.5.3/vendor/bundle/jruby/2.3.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/client.rb:131:in `perform_request'
/home/abc/elk/logstash-6.5.3/vendor/bundle/jruby/2.3.0/gems/elasticsearch-api-5.0.5/lib/elasticsearch/api/actions/search.rb:183:in `search'
/home/abc/elk/logstash-6.5.3/vendor/bundle/jruby/2.3.0/gems/logstash-input-elasticsearch-4.2.1/lib/logstash/inputs/elasticsearch.rb:200:in `do_run'
/home/abc/elk/logstash-6.5.3/vendor/bundle/jruby/2.3.0/gems/logstash-input-elasticsearch-4.2.1/lib/logstash/inputs/elasticsearch.rb:188:in `run'
/home/abc/elk/logstash-6.5.3/logstash-core/lib/logstash/pipeline.rb:409:in `inputworker'
/home/abc/elk/logstash-6.5.3/logstash-core/lib/logstash/pipeline.rb:403:in `block in start_input'
```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 16, 2019, 12:49pm UTC](https://discuss.elastic.co/t/unable-to-retreive-data-from-index-using-elasticsearch-input-plugin-in-logstash/181375/2 "2019-05-16T12:49:01Z")

</div>

> [@Vishnu\_mk](#):
>
> ```
> <ADDRESS>
> Generated Thu, 16 May 2019 09:00:09 GMT by TGPROXY-Backup (squid/2.6.STABLE6)
> </ADDRESS>
> 
> ```

There is a squid proxy between you and elasticsearch. Is there any indication in the elasticsearch logs that the request is reaching it?

---

<div class="post-metadata">

**Author:** ![Vishnu\_mk](https://avatars.discourse-cdn.com/v4/letter/v/71c47a/32.png) [@Vishnu\_mk](https://discuss.elastic.co/u/Vishnu_mk)\
**Post date:** [May 16, 2019, 1:36pm UTC](https://discuss.elastic.co/t/unable-to-retreive-data-from-index-using-elasticsearch-input-plugin-in-logstash/181375/3 "2019-05-16T13:36:13Z")

</div>

No there is no indication in elasticsearch logs

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 16, 2019, 1:57pm UTC](https://discuss.elastic.co/t/unable-to-retreive-data-from-index-using-elasticsearch-input-plugin-in-logstash/181375/4 "2019-05-16T13:57:36Z")

</div>

That makes me think the problem is in the proxy. Can you not bypass the proxy?

---

<div class="post-metadata">

**Author:** ![Vishnu\_mk](https://avatars.discourse-cdn.com/v4/letter/v/71c47a/32.png) [@Vishnu\_mk](https://discuss.elastic.co/u/Vishnu_mk)\
**Post date:** [May 16, 2019, 4:54pm UTC](https://discuss.elastic.co/t/unable-to-retreive-data-from-index-using-elasticsearch-input-plugin-in-logstash/181375/5 "2019-05-16T16:54:10Z")

</div>

Thanking for you reply but i didnt understand what bypassing proxy mean

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 16, 2019, 5:10pm UTC](https://discuss.elastic.co/t/unable-to-retreive-data-from-index-using-elasticsearch-input-plugin-in-logstash/181375/6 "2019-05-16T17:10:05Z")

</div>

You are connecting to 172.25.8.45:9200 and your post suggests that is a Squid proxy, not an elasticsearch instance. I am suggesting you should connect directly to elasticsearch, not to squid.

---

<div class="post-metadata">

**Author:** ![Vishnu\_mk](https://avatars.discourse-cdn.com/v4/letter/v/71c47a/32.png) [@Vishnu\_mk](https://discuss.elastic.co/u/Vishnu_mk)\
**Post date:** [May 16, 2019, 5:11pm UTC](https://discuss.elastic.co/t/unable-to-retreive-data-from-index-using-elasticsearch-input-plugin-in-logstash/181375/7 "2019-05-16T17:11:51Z")

</div>

Ok understood thank you so much

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 13, 2019, 5:11pm UTC](https://discuss.elastic.co/t/unable-to-retreive-data-from-index-using-elasticsearch-input-plugin-in-logstash/181375/8 "2019-06-13T17:11:56Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.