How to use ruby variable outside ruby block?

Elastic Stack Logstash
1

Hi,

I like to know, how to use ruby variable outside ruby code block.

say for eg.

input {
file {
path => "/logfile/*"
start_position => "beginning"
sincedb_path => "/dev/null"
}
}

filter {

    ruby {
        code => "
            log_pattern = event['fields'].split(',')
            log_pattern.each_index { |i| 
                break if ['message'] == /log_pattern[i]/
                    event['index_value'] = log_pattern[i]
            }
        "
    }

}

output {
elasticsearch {
hosts=> [localhost:9200]
index => index_value
}
stdout
{
codec=>rubydebug
}
}

I want to use index value in output plugin as well as in filter plugin. how to do that?

Thanks,

2

It is not clear to me what you are trying to do, but if you want to set the value of a field on an event, you should use event.set, which is documented here.

I think in old versions you could do this

event['index_value'] = log_pattern[i]

but now you would would have to do

event.set('index_value', log_pattern[i])

And in your output, index => index_value, as documented under sprintf format here, should be

index => "%{index_value}"
3

Thanks for responding. But the index_value is not applied in output plugin. In elasticsearch output, index => "%{index_value}" the actual value is not applied in logstash output index tag.

input {
file {
path => "/digital/*"
start_position => "beginning"
sincedb_path => "/dev/null"
}
}

filter {

    ruby {
        code => "
            log_pattern = event['{{dvps.fields}}'].split(',')
            log_pattern.each_index { |i| 
                break if ['message'] == /log_pattern[i]/
                    event.set('index_value', log_patter[i])
            }
        "
    }

}

output {
elasticsearch {
hosts=> ["localhost:9200"]
index => "%{index_value}"
}
stdout
{
codec=>rubydebug
}
}

Thanks.

4

What do you see in the rubydebug output for an event?

5

Ruby Exception.

"tags" => [
[0] "_rubyexception"
]

6

OK, so in your logstash logfile there should be an error message. Possibly something like "undefined variable or method 'log_patter'". :smiley:

7

yep. you are right dude. But is there any possible to parse log files in ruby filter similar to kv filter?

8

What does your input line look like, and what do you want the output event to look like?

9

say for example this is my sample log,
2018-05-30 11:00:04,355 [INFO] packagename=sample.package.group, date=10-12-2017, status=pending,...etc.

I want to split this message with dynamic key value like packagename,data,status. and also the log format should not be same as all the files. so I have to split this logs with some respective keywords like how kv filter works. how to do that in ruby filter with dynamic key values.

10

I would use dissect (or grok) to take the date and log level off, then use a kv filter to parse the rest of the line. Why do you want to do it in ruby?

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.