Using Regex in Yaml for Logstash Translate Filter

Elastic Stack Logstash
1

Good day all, I need some help.

I am trying to use the Logstash Translate Filter to enrich network data that bro is generating and that I'm ingesting into my ELK stack. For example, here is how i was enriching data manually:

translate {
    field => "id.orig_h"
    destination => "src_comp_name"
    dictionary => [
        "192.168.1.1", "Home_Router",
        "192.168.1.150", "My_Laptop",
        "192.168.1.210", "My_Desktop"
        ]
}

While this works, it doesn't scale for what I am going to eventually need it for. So I'm trying to move my dictionary to a yaml file and use regex to match IP addresses to assign them tags. so I edited my translate function to:

translate {
    field => "id.orig_h"
    destination => "src_comp_name"
    dictionary_path => '/etc/logstash/config/compNames.yaml'
}

Below is the contents of roughly what I want to do in the Yaml file:

'^192\.168\.1\.[1-2]$': "Home_Routers"
'^192\.168\.1\.1[0-9]{2}$': "Home_Laptops"
'^192\.168\.1\.2[0-9]{2}$': "Home_Desktops"

This would cause 192.168.1.1/2 to be tagged as routers, anything in the .100-199 range to be tagged as Home_Laptops, and anything from 200-255 to be tagged as "Home_Desktops". I have tried multiple ways of using regex in the Yaml file, but I'm either getting errors like "LogStash::Filters::Translate: can't convert Array into Hash when loading dictionary file at /etc/logstash/config/compNames.yaml", or logstash is correctly starting but not tagging traffic that should be matching.

Any guidance out there on how to implement regex matching in a Yaml file for data enrichment via Logstash Translate Filter?

2

For posterity's sake, I just got the answer to my question here: http://stackoverflow.com/questions/41522280/how-to-use-regex-in-yaml-file-for-logstash-translate-filter

3

Yep, also see https://www.elastic.co/guide/en/logstash/5.1/plugins-filters-translate.html#plugins-filters-translate-regex

1 Like
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.