CSV filter in logstash question

Elastic Stack Logstash
1

How would you add a tag to a column that has been parsed by the CSV plugin?

For instance, I have the following and it gets all the columns indexed, but it does not add tags to the matching [sourcetype] regex.

filter {
  if [type] == "csv" {
    csv {
      separator => ","
      columns => ["timestamp", "source", "sourcetype", "host", "index", "raw"]
      remove_field => ["message"]
      add_tag => ["csv"]
    }
    grok {
      match => ["timestamp", "%{TIMESTAMP_ISO8601:timestamp} %{WORD:timezone}"]
      overwrite => ["timestamp"]
    }
    date{
      #match => ["timestamp", "YYYY-MM-dd HH:mm:ss,SSS"]
      match => ["timestamp", "ISO8601"]
    }
    if [sourcetype] =~ /^vpc$/ {
      mutate {
        add_tag => ["vpc"]
      }
    }
    if [sourcetype] =~ /^iis$/ {
      mutate {
        add_tag => ["iis"]
      }
    }
  }
}
2

If you are anchoring both the beginning and end of a string I think it would be clearer to use a simple string equality test.

if [sourcetype] == "vpc" {
    [...]
3

The sourcetype column can have multiple entries for the string vpc, for instance it could be vpc1, vpc2... So I though I would have to regex match it?

4

Yes, but the ^ and $ anchor the regexp to start of string and end of string, so ^vpc$ only matches the exact string vpc.

If you want any string containing vpc use /vpc/. If you want any string beginning with vpc use /^vpc/.

5

Umm, yeah... Wow, it's been a long day. Thanks Badger for the sanity check!

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.