Ruby time filter on LS 1.5

Elastic Stack Logstash
1

I want that logstash drop events older than 6 hours, for that purpose I have this ruby filter:

ruby {
   code => "event.cancel if Time.now - event['@timestamp'] > 21600"
}

This filters works like a charm in this environment:

  • ubuntu 12
  • logstash 1.4.2 (installed by the elastic repository)
  • logstash-forwarder 0.3

I'm trying to upgrade to logstash 1.5 and logstash-forwarder 0.4. It seems like everything works fine but if I add those specific ruby filter the logstash-forwarder start to print this lines:

2015/05/29 08:54:29.573112 Read error looking for ack: read tcp x.x.x.:6782: i/o timeout
2015/05/29 08:54:29.573251 Setting trusted CA from file: /etc/ssl/certs/logstash-forwarder.crt
2015/05/29 08:54:29.574457 Connecting to [x.x.x.x]:6782 (ls01.server.com)
2015/05/29 08:54:29.656897 Connected to x.x.x.x
2015/05/29 08:54:44.657309 Read error looking for ack: read tcp x.x.x.x:6782: i/o timeout
2015/05/29 08:54:44.657458 Setting trusted CA from file: /etc/ssl/certs/logstash-forwarder.crt
2015/05/29 08:54:44.658634 Connecting to [x.x.x.x]:6782 (ls01.server.com)
2015/05/29 08:54:44.724829 Connected to x.x.x.x
2015/05/29 08:54:59.725106 Read error looking for ack: read tcp x.x.x.x:6782: i/o timeout
2015/05/29 08:54:59.725255 Setting trusted CA from file: /etc/ssl/certs/logstash-forwarder.crt
2015/05/29 08:54:59.726469 Connecting to [x.x.x.x]:6782 (ls01.server.com)
2015/05/29 08:54:59.800727 Connected to x.x.x.x

The environment where this filter failed is:

  • ubuntu 14
  • logstash 1.5 (elastic repository)
  • logstash-forwarder 0.4 (elastic repository)

Somebody has the same issue?

2

Another thing, if I add that filter to the logstash configuration file, I'm not able to stop the daemon. I have to kill it with "-9":

root@ls01:~# service logstash stop
Killing logstash (pid 10865) with SIGTERM
Waiting logstash (pid 10865) to die...
Waiting logstash (pid 10865) to die...
Waiting logstash (pid 10865) to die...
Waiting logstash (pid 10865) to die...
Waiting logstash (pid 10865) to die...
logstash stop failed; still running.
root@ls01:~#

Nevertheless, it that filter is not present, the logstash stops ok:

root@ls01:~# service logstash stop
Killing logstash (pid 10727) with SIGTERM
Waiting logstash (pid 10727) to die...
Waiting logstash (pid 10727) to die...
logstash stopped.
root@ls01:~#
3

Can you attach your full logstash config? Can you also tell me what makes you think the ruby filter you are using is to blame?

4

Hi Jordan,

I tried replacing "Time.now" with "LogStash::Timestamp.now" and it works!

Thanks!!