S3 Multiple outputs using Logstash and Winlogbeat

Elastic Stack Logstash
1

Hello everybody,
I'm using AWS with a Windows EC2 instance and I collect logs by using Logstash and Winlogbeat. I'm able to collect all logs if I keep all together.
However, I would like to separate logs according if are Application, Security or System but the if condition doesn't seem to be working. I don't get any errors btw, I simply don't get logs.
This is my logstash.conf:

input {
    beats {
        port => 5044
    }
}

output {
    if [tags] == "winlogbeat-application" {
        s3 {
            region => "us-west-1"
            bucket => "s3-mybucket"
            prefix => "/AWSLogs/Application"
            role_arn => "arn:xxxxx"
            rotation_strategy => "time"
            time_file => 10
            validate_credentials_on_root_bucket => false
            canned_acl => "bucket-owner-full-control"
        }   
    } else if [tags] == "winlogbeat-system" {
        s3 {
            region => "us-west-1"
            bucket => "s3-mybucket"
            prefix => "/AWSLogs/System"
            role_arn => "arn:xxxxx"
            rotation_strategy => "time"
            time_file => 10
            validate_credentials_on_root_bucket => false
            canned_acl => "bucket-owner-full-control"
        }
    } else if [tags] == "winlogbeat-security" {
        s3 {
            region => "us-west-1"
            bucket => "s3-mybucket"
            prefix => "/AWSLogs/Security/"
            role_arn => "arn:xxxxx"
            rotation_strategy => "time"
            time_file => 10
            validate_credentials_on_root_bucket => false
            canned_acl => "bucket-owner-full-control"
        }
    } else {
        s3 {
            region => "us-west-1"
            bucket => "s3-mybucket"
            prefix => "/AWSLogs/Other/"
            role_arn => "arn:xxxxx"
            rotation_strategy => "time"
            time_file => 10
            validate_credentials_on_root_bucket => false
            canned_acl => "bucket-owner-full-control"
        }
    }
}

This is my winlogbeat.yml file:

winlogbeat.event_logs:
  - name: Application
    ignore_older: 72h
	  tags: ["winlogbeat-application"]

  - name: System
    tags: ["winlogbeat-system"]

  - name: Security
    tags: ["winlogbeat-security"]

# ================================== Outputs ===================================

# Configure what output to use when sending the data collected by the beat.

# ------------------------------ Logstash Output -------------------------------
output.logstash:
  # The Logstash hosts
  hosts: ["localhost:5044"]

# ================================= Processors =================================
processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_cloud_metadata: ~

# ================================== Logging ===================================

# Sets log level. The default log level is info.
# Available log levels are: error, warning, info, debug
logging.level: debug

# At debug level, you can selectively enable logging only for some components.
# To enable all selectors, use ["*"]. Examples of other selectors are "beat",
# "publisher", "service".
logging.selectors: ["*"]

The issue seems to be the if condition because I tried modifying logstash.conf and collecting just Application logs and if I use the if expression I don't get anything but if I remove that I do get logs. Any help will be appreciated. thank you in advance.

Regards,
Tizi

2

The tags field is the array. Try:
if "winlogbeat-application" in [tags] { ...

Other conditions should be changed as well.

3

Hello @Rios , thank you for your reply. I did the changes suggested and now works! I really appreciated, thank you !

Regards,
Tizi

1 Like
4

You welcome. Long live the king and the Elastic team.

1 Like