Distinguish Winlogbeat traffic in Logstash

[giuliano (pellegrini)]

HI all.....I use ELK 6.2.3 with Winlogbeat and Auditbeat for Linux.

I have 20 Web Server IIS and I want to send all audit logs to Elasticsearch in only one index (for example web-2018.04.26). The Elasticsearch records another data as Palo Alto log, NGINX log ecc...so I would like to discern all Winlogbeat log from another traffic for create a dedicate web index for all IIS machines.

I have try with "tags", "fields", "type" in input filter but doesn't running successfully. There are a two simple tests.

1° test:

input
{
  beats
  {
    port => 5003
    type => "web"
  }
}

output
{
  if [type] == "web"
  {
    elasticsearch
    {
      hosts => "localhost:9200"
      index => "web-%{+YYYY.MM.dd}"
    }
  }
}

2° test: with tags in winlogbeat.yml

input
{
  beats
  {
    port => 5003
  }
}

output
{
  if [tags] == "web"
  {
    elasticsearch
    {
      hosts => "localhost:9200"
      index => "web-%{+YYYY.MM.dd}"
    }
  }
}

this is a piece of winlogbeat.yml:

winlogbeat.event_logs:
  - name: Security
    tags: ["web"]
    event_id: 4624, 4625
output.logstash:
  # The Logstash hosts
  hosts: ["<IP>:5003"]

Any idea for fix this situation...if is possible.....?

Thanking in advance and sorry for my english.

Best Regards

Giuliano

[andrewkroh]

This can be accomplished with tags, but combining data with different mappings in the same index leads to sparsity (see https://www.elastic.co/blog/index-vs-type). An alternative could be to have different indices with a common prefix to allow for a single index pattern (web-*) in Kibana that covers them all (like web-winlogbeat-2017.04.27 and web-auditbeat-2017.04.27).

Also be advised that you must manage the index templates on your own when going this route of combining data from different beats in the same index.

Here's the example you were looking for:

winlogbeat.event_logs:
  - name: Security
    tags: ["web"]
    event_id: 4624, 4625

output
{
  if "web" in [tags] {
    // output to Elasticsearch
  }
}

See https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html for more example of conditionals in Logstash.