Logstash Persistent Variables

Elastic Stack Logstash
1

I wanted to create a persistent boolean that I can change whenever a certain message appears. Based on this boolean I would change my output{ elastic search{ index file reference}}.
E.g.
log GET
log dog
Log PUT
log 1234
The boolean should be set to true when it sees a 'log GET', and parses the 'log dog' with index config 1. When I receive 'log PUT', the boolean should be set to false and parse the 'log 1234' with index config 2. I would really appreciate any help on this issue! Thanks

2

You can use a ruby filter as described here. It requires '--pipeline.workers 1' and pipeline.java_execution has to be false until this bug is fixed.

3

Do you know how to declare a boolean, and if you use the ruby filter or not? Sorry, I am very new to Ruby...

4

Something like this

    ruby {
        init => '@b = false'
        code => '
            m = event.get("message")
            if m and m =~ /log get/i
                @b = true
            end
            if m and m =~ /log put/i
                @b = false
            end
            event.set("someField", @b)
        '
    }

will add a field [someField] which will be a boolean.

5

Thanks so much for that. My issue is that how do I reference that field after I receive the next log. So when I get 'log dog' how do I reference the someField event from the preceding log. (In this case, 'log get' ALWAYS precedes 'log dog'. Once again, thanks a log

When I reference in output:
if event.get("someField") == 'true' I receive an error

edit: added example error

6 
if [someField] {
...
}
7

Ok it is almost working. When I do
mutate{
add_field => {"someField" => "event.get("someField")"}
}
I am getting an issue. How do I set someField = to the event.get(someField) rather than the string "event.get(someField)"? Thanks so much badger!

8

The documentation includes examples of sprintf references in add_field.

9

Yeah I took a look at that, however it was still not working even when I did
"%{event.get(someField)}".
I believe the reason for this is because perhaps the event.set("someField") is NOT persistent? And it is not able to see the last update to @put.
E.g.
log GET
@b is set to false
event.set(someField)
log dog
(I do not set any boolean value, I want to reference whatever value the boolean was last)

If you can help me solve this I will love you forever lmao

10

event.set and event.get only work in ruby filters. In the rest of the logstash configuration you would reference a field as [someField], and in a sprintf reference as %{[someField]}

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.