# Logstash ruby filter issues with HTTP queries

**URL:** <https://discuss.elastic.co/t/logstash-ruby-filter-issues-with-http-queries/180017>\
**Category:** Logstash\
**Created:** [May 7, 2019, 3:40pm UTC](https://discuss.elastic.co/t/logstash-ruby-filter-issues-with-http-queries/180017 "2019-05-07T15:40:28Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![ayesyev](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ayesyev/32/45744_2.png) [@ayesyev](https://discuss.elastic.co/u/ayesyev)\
**Post date:** [May 7, 2019, 3:40pm UTC](https://discuss.elastic.co/t/logstash-ruby-filter-issues-with-http-queries/180017/1 "2019-05-07T15:40:28Z")

</div>

Hi there!  
I'm stuck with the issue, I have with an http client in my ruby filter in Logstash 6.5.4.

Here is a snippet of the filter code

```
require 'net/http'
require 'uri'
require 'json'

def register(params)
  @http = Net::HTTP.new("http://dcaserver", 8080)
  @log = Logger.new("xfecats.log", 5, 10240000)
  @log.level = Logger::DEBUG
end

def filter(event)
...
        begin
          response = @http.request ( Net::HTTP::Get.new("/url/mobile/#{domain}") )
          if response.is_a? Net::HTTPOK
            json_string = response.body
            json = JSON.parse(json_string)
            json_cats = json['value']
            if json_cats.nil?
              new_event.set("[xfe][category]", "Unknown")
              return[new_event]
            end
            ...
        rescue Exception => ex
          @log.debug "Exception querying DCA for domain #{domain} :#{ex.message}\n#{ex.backtrace}"
        end
        return[new_event]
.....
end

```

So, basically, my filter makes HTTP request to some local service. There are could be many of them. What I noticed, that almost 90% of them are failing with strange errors, like these three:

```
D, [2019-05-07T14:39:39.077899 #1] DEBUG -- : Exception querying DCA for domain google.cf :undefined method `+' for nil:NilClass
["uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/protocol.rb:212:in `write0'",
"uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/protocol.rb:185:in `block in write'",
"uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/protocol.rb:202:in `writing'",
"uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/protocol.rb:184:in `write'",
"uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http/generic_request.rb:334:in `write_header'",
"uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http/generic_request.rb:127:in `exec'",
"uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:1446:in `block in transport_request'",
"org/jruby/RubyKernel.java:1114:in `catch'",
"uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:1445:in `transport_request'",
"uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:1418:in `request'",
"uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:1411:in `block in request'",
"uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:858:in `start'",
"uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:1409:in `request'",
"/usr/share/logstash/xfecategories.rb:127:in `block in filter'", "org/jruby/RubyArray.java:1734:in `each'",
"/usr/share/logstash/xfecategories.rb:97:in `filter'",
"/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-ruby-3.1.5/lib/logstash/filters/ruby/script/context.rb:55:in `execute_filter'",...

D, [2019-05-07T14:39:39.327289 #1] DEBUG -- : Exception querying DCA for domain qtnet.ad.jp :wrong status line: "Access-Control-Allow-Origin: *"
["uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http/response.rb:42:in `read_status_line'",
"uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http/response.rb:29:in `read_new'",
"uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:1448:in `block in transport_request'",
"org/jruby/RubyKernel.java:1114:in `catch'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:1445:in `transport_request'",
"uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:1418:in `request'",
"uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:1411:in `block in request'",
"uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:858:in `start'",
"uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:1409:in `request'",
"/usr/share/logstash/xfecategories.rb:127:in `block in filter'", "org/jruby/RubyArray.java:1734:in `each'",
"/usr/share/logstash/xfecategories.rb:97:in `filter'",...

D, [2019-05-07T14:39:39.180247 #1] DEBUG -- : Exception querying DCA for domain ghnet.com.br :undefined method `closed?' for nil:NilClass
Did you mean? clone
["uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:1486:in `begin_transport'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:1444:in `transport_request'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:1418:in `request'", "/usr/share/logstash/xfecategories.rb:127:in `block in filter'", "org/jruby/RubyArray.java:1734:in `each'", "/usr/share/logstash/xfecategories.rb:97:in `filter'", ...

```

After some googling, I figured out that these issues might be connected the use of WebMock gem, see [this](https://github.com/rails/webpacker/issues/822). But I don't intentially use WebMock and I can't see that Logstash does.

Does anyone have any piece of advice?  
I'm completely stuck here...

Thanks!

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 7, 2019, 3:53pm UTC](https://discuss.elastic.co/t/logstash-ruby-filter-issues-with-http-queries/180017/2 "2019-05-07T15:53:42Z")

</div>

Does setting "--pipeline.workers 1" eliminate the errors? If it does you have thread-safety issues.

---

<div class="post-metadata">

**Author:** ![ayesyev](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ayesyev/32/45744_2.png) [@ayesyev](https://discuss.elastic.co/u/ayesyev)\
**Post date:** [May 7, 2019, 5:26pm UTC](https://discuss.elastic.co/t/logstash-ruby-filter-issues-with-http-queries/180017/3 "2019-05-07T17:26:14Z")

</div>

It did help.

Thanks a lot!

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 7, 2019, 7:31pm UTC](https://discuss.elastic.co/t/logstash-ruby-filter-issues-with-http-queries/180017/4 "2019-05-07T19:31:36Z")

</div>

BTW, you know there is an [http](https://www.elastic.co/guide/en/logstash/current/plugins-filters-http.html) filter plugin, right?

---

<div class="post-metadata">

**Author:** ![ayesyev](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ayesyev/32/45744_2.png) [@ayesyev](https://discuss.elastic.co/u/ayesyev)\
**Post date:** [May 7, 2019, 7:47pm UTC](https://discuss.elastic.co/t/logstash-ruby-filter-issues-with-http-queries/180017/5 "2019-05-07T19:47:13Z")

</div>

No, I didn't. Thanks for pointing out!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 4, 2019, 7:47pm UTC](https://discuss.elastic.co/t/logstash-ruby-filter-issues-with-http-queries/180017/6 "2019-06-04T19:47:18Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.