Ruby Script in Logstash :: Enters a NULL into the Data?

[redapplesonly (RAO)]

Hi Logstash Jedi Masters,

I’m running Logstash 7.7.1 in the Docker container. (Yes, I know I need to upgrade) In my Logstash config file, I’m running a snippet of Ruby code:

  ruby {
    init => "
      require 'socket'
    "

    code => '
      socket = TCPSocket.new("192.168.3.1", 12345)
      socket.write (event.to_hash).to_s
      readable = IO.select [socket], nil, nil, 0.1
      if readable then
        response = socket.recv(1000)
      else
        response = "UNAVAILABLE"
      end
      event.set("nDPI_Application", response)
      socket.close
    ' 
  }

When a data record arrives in my Logstash, the code is supposed to do the following: Open a TCP socket to a remote host, send the entire event to the host, then wait 100 msec to get a response back. The remote host should send back a single string. If the remote host replies in time, the host’s response is added to a new field called RemoteHostInfo. However, if no response is received, then Logstash should populate the RemoteHostInfo field with “UNAVAILABLE”

(I am not a Ruby coder, and the code has been pasted together with a lot of trial-and-error.)

My Logstash exports all data to an Elasticsearch instance, so I can monitor the final data records there. The good news is that when the remote host is available, I can see that the above Ruby code works great. However, when the remote host is down, I see this in the records:

sql> select HostA, HostB, RemoteHostInfo, sum( totalBytes )
>   from \"myIndex\"
>   where \"@timestamp\" >= NOW()- INTERVAL 10 MINUTES
>   group by HostA, HostB, RemoteHostInfo;
  HostA     |  HostB     | RemoteHostInfo | sum( totalBytes )
------------+------------+----------------+------------------
10.10.10.50 |10.10.10.100|null            |1378304.0
10.10.10.51 |10.10.10.110|null            |4.95383552E9
10.10.10.52 |10.10.10.120|null            |4.843754496E9
10.10.10.53 |10.10.10.130|null            |5.076754432E9
10.10.10.54 |10.10.10.140|null            |2.46971392E8
10.10.10.55 |10.10.10.150|null            |1.53665536E8 

Hmm. So when the remote host is not reachable, Logstash is populating the RemoteHostInfo field with NULL, not “UNAVAILABLE”. That’s not necessarily a bad thing, but it does create data automation problems further downstream in my pipeline.

What I can’t tell is if this is a Ruby problem or a Logstash problem. And unfortunately, I don’t know how to troubleshoot the Ruby code when its within the Docker container. Any tips? Thank you.