Logstash + Redis

Hi guys,
I am trying to figure out how to use Logstash with Redis
Does anyone know how to configure logstash to insert a specific field in message in Redis ?

For example, I have a json message with some fields, like: Name, Passport Number, Age, ..
I would like to insert in Redis (using redis output plugin) only the Passport Number (as Key) and Name (as Value)

Simply use the mutate filter remove processor and remove the fields you do not want to send to redis

A prune filter with a whitelist_names option is another alternative.

1 Like

@stephenb @Badger
I cant do it.
There is other ouput where i need to send all the information.
If i use mutate filter, all outputs will be affected.

Use pipeline-to-pipeline communication with a forked-path pattern.

2 Likes

Nice, i will take a look in your suggestion.

I think that you can also use the codec option in the redis output to format the output message.

Something like this:

codec => plain { format => "%{field1} %{field2} %{fieldN}" }

But if you have different outputs, I would say that the pipeline-to-pipeline communication is the best approach.

Just to update, the solution i found to my case was put the following lines in the filter:

  • to insert data in Redis (inserting data from diameter_Session-Id and msisdn fields):
                if "3868" in [tcp_srcport] {
                    ruby {
                        init => 'require "redis"; $rc = Redis.new(host: "127.0.0.1", port: 6379)'
                        code => '
                            $rc.sadd(event.get("diameter_Session-Id"), event.get("msisdn"))
                            $rc.expire(event.get("diameter_Session-Id"), 3600)
                        '
                    }
                }
  • to get data in Redis (retrieving data from Redis using the value of diameter_Session-Id field as key):
                if "3868" in [tcp_dstport] {
                    ruby {
                        init => 'require "redis"; $rc = Redis.new(host: "127.0.0.1", port: 6379)'
                        code => 'event.set("msisdn_ruby", $rc.smembers(event.get("diameter_Session-Id")))'
                    }
                }

A @@class variable should work. I don't think you ever need to use a $global variable.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.