Ruby Filter output to Field


(Josh) #1

I'm testing a ruby filter to build a small list of hosts that are producing less than a certain number of events per minute.

Is there a standard method to get this output back into the event json?

Currently I'm trying with the following code. The first line generates the list of nodes, the second one should insert it into the message field.

count = event.select{ |key| key[/app\.count$/] }.select{ |key, value| value.to_s.to_i <= 8 }.to_s.gsub('events.', '').gsub('.count', '')
event['message'].insert(0, var) 

However this doesn't appear to be working. It's not required that the list goes into the message field. Really I just need to get it to logstash's output. Does anyone have any ideas?

I apologize ahead of time, I'm not very experienced with ruby. I appreciate any help.


(Rafał Trójniak) #2

Can you provide example event that your script should work on ?


(Josh) #3

I actually figured out part of what I was doing wrong. My conditional was incorrect so the logs were passing right over the filter.

Now however I get the new field created with brackets but no content.

Current ruby filter is

event['inactive'] = event.to_s.scan(/\w+\-app\.count...[0-4]?[0-9]/).to_s.gsub(/\.count/, ' ')

Below is an example event

  {
  "_index": "logstash-2015.06.02",
  "_type": "logs",
  "_id": "AU22Wu6WI9PjSCKuhHlk",
  "_score": null,
  "_source": {
    "@version": "1",
    "@timestamp": "2015-06-02T22:18:41.352Z",
    "message": "syslog",
    "events.esd1-qa2.count": 2,
    "events.esd1-qa2.rate_1m": 0.021746301622157297,
    "events.trello-qa2.count": 266,
    "events.trello-qa2.rate_1m": 2.9085596738152457,
    "events.pg1-qa2.count": 2,
    "events.pg1-qa2.rate_1m": 0.021746301622157297,
    "events.haproxy1-app.count": 1,
    "events.haproxy1-app.rate_1m": 0.010541994097562062,
    "events.trello-app.count": 6,
    "events.trello-app.rate_1m": 0.07707065958795782,
    "events.cc4-app.count": 49,
    "events.cc4-app.rate_1m": 1.043409992451638,
    "events.cc2-qa2.count": 940,
    "events.cc2-qa2.rate_1m": 17.067786217080286,
    "events.pg2-qa2.count": 2,
    "events.pg2-qa2.rate_1m": 0.010541994097562062,
    "events.jira-app.count": 8,
    "events.jira-app.rate_1m": 0.10276087945061044,
    "events.office365-qa1.count": 214,
    "events.office365-qa1.rate_1m": 14.321784645741761,
    "events.cc4-qa2.count": 916,
    "events.cc4-qa2.rate_1m": 20.449810405258265,
    "events.cc3-qa2.count": 899,
    "events.cc3-qa2.rate_1m": 9.76185652915327,
    "events.haproxy1-qa2.count": 6,
    "events.haproxy1-qa2.rate_1m": 0.07157574646541,
    "events.esd1-qa1.count": 8,
    "events.esd1-qa1.rate_1m": 0.09017685087937428,
    "tags": [
      "metric",
      "syslog-overflow"
    ]
  }
}

Output is

"inactive': []

(system) #4