Logstash .Conf file

Elastic Stack Logstash
1

I am trying to collect logs from multiple inputs and write them into 2 indices.

Please refer the below conf file, am able to create a index called syslog but am not able to create index called winlog. All the logs are going to syslog index.

input {
  beats {
    port => 5044

  }
}


output {
        if [beats][hostname] == "DESKTOP-ONEXXXXX"
                {
                elasticsearch {
                        hosts => "192.168.77.137:9200"
                        index => "winlog-%{+YYYY.MM.dd}"
                        user => "elastic"
                        password => "********"
                        }
                }
        else{
                elasticsearch {
                        hosts => "192.168.77.137:9200"
                        index => "syslog-%{+YYYY.MM.dd}"
                        user => "elastic"
                        password => "********"
                        }

                }
}
2

You probably want to test [beat][hostname], not [beats][hostname].

3

I have tried [beat] as well, no luck

4

@andrewkroh Could you please have a look at this?

5

The field name is beat.hostname not beats.hostname.

The conditiona is pretty simple, if the document ended up in the syslog indice, probably the hostname is not what you are testing.

Share an example of a document that should be in the index winlog-* but ended up in the index syslog-*.

6

I get an error for beat.hostname, so I tried [beat][hostname] but still no luck. The logs are not being ingested.

output {
if [beat][hostname] == "DESKTOP-ONE7L9S"
                {
                elasticsearch {
                        hosts => "192.168.77.137:9200"
                        index => "winlog-%{+YYYY.MM.dd}"
                        user => "elastic"
                        password => "********"
                        }
                }
 else{
                elasticsearch {
                        hosts => "192.168.77.137:9200"
                        index => "syslog-%{+YYYY.MM.dd}"
                        user => "elastic"
                        password => "********"
                        }

                }
}
7

As I said, you need to share an example of a message that is not working.

8

This is the example document, I am not receiving these logs on either of the indices now.

{
  "_index": "winlog-2021.07.09",
  "_type": "_doc",
  "_id": "1G5tinoBZ1pjQ41D047O",
  "_score": 1,
  "fields": {
    "agent.version.keyword": [
      "7.13.2"
    ],
    "event.category": [
      "iam"
    ],
    "host.name.keyword": [
      "DESKTOP-ONE7L9S"
    ],
    "event.outcome.keyword": [
      "success"
    ],
    "host.hostname": [
      "DESKTOP-ONE7L9S"
    ],
    "host.mac": [
      "8c:dc:d4:83:2a:e2",
      "00:50:56:c0:00:01",
      "00:50:56:c0:00:08",
      "f8:16:54:58:38:d3"
    ],
    "winlog.process.pid": [
      72
    ],
    "winlog.event_data.TargetDomainName.keyword": [
      "DESKTOP-ONE7L9S"
    ],
    "host.os.version": [
      "10.0"
    ],
    "winlog.logon.id": [
      "0x3e7"
    ],
    "agent.name": [
      "DESKTOP-ONE7L9S"
    ],
    "event.outcome": [
      "success"
    ],
    "host.os.type": [
      "windows"
    ],
    "user.id": [
      "S-1-5-18"
    ],
    "agent.id.keyword": [
      "2221956b-cecb-431c-a9fe-7815e64305a1"
    ],
    "agent.hostname": [
      "DESKTOP-ONE7L9S"
    ],
    "tags": [
      "Windowslog",
      "beats_input_codec_plain_applied"
    ],
    "related.user": [
      "DESKTOP-ONE7L9S$",
      "ashwi"
    ],
    "host.architecture": [
      "x86_64"
    ],
    "event.provider": [
      "Microsoft-Windows-Security-Auditing"
    ],
    "winlog.event_data.CallerProcessName.keyword": [
      "C:\\Windows\\System32\\svchost.exe"
    ],
    "event.code": [
      "4798"
    ],
    "agent.id": [
      "2221956b-cecb-431c-a9fe-7815e64305a1"
    ],
    "winlog.provider_name.keyword": [
      "Microsoft-Windows-Security-Auditing"
    ],
    "winlog.event_data.SubjectUserSid": [
      "S-1-5-18"
    ],
    "winlog.api.keyword": [
      "wineventlog"
    ],
    "winlog.process.thread.id": [
      700
    ],
    "tags.keyword": [
      "Windowslog",
      "beats_input_codec_plain_applied"
    ],
    "user.name": [
      "DESKTOP-ONE7L9S$"
    ],
    "winlog.event_data.CallerProcessId": [
      "0x15b4"
    ],
    "winlog.event_data.CallerProcessName": [
      "C:\\Windows\\System32\\svchost.exe"
    ],
    "host.ip": [
      "fe80::144f:79e0:7649:9d5e",
      "169.254.157.94",
      "fe80::8d99:9241:be7e:3a48",
      "192.168.58.1",
      "fe80::a0c3:ece7:1118:2a78",
      "192.168.77.1",
      "2001:bb6:8266:fc58:785c:a917:280a:f49f",
      "2001:bb6:8266:fc58:b97a:b0c4:70d7:f130",
      "fe80::785c:a917:280a:f49f",
      "192.168.1.12"
    ],
    "agent.type": [
      "winlogbeat"
    ],
    "event.type.keyword": [
      "user",
      "info"
    ],
    "host.os.kernel.keyword": [
      "10.0.19041.1083 (WinBuild.160101.0800)"
    ],
    "winlog.event_data.TargetSid": [
      "S-1-5-21-495902564-1190171751-3348208613-1001"
    ],
    "winlog.event_data.SubjectLogonId": [
      "0x3e7"
    ],
    "winlog.api": [
      "wineventlog"
    ],
    "event.provider.keyword": [
      "Microsoft-Windows-Security-Auditing"
    ],
    "user.domain": [
      "WORKGROUP"
    ],
    "host.id": [
      "76e7c87e-b848-4546-a0ab-8edd66c1cd33"
    ],
    "agent.type.keyword": [
      "winlogbeat"
    ],
    "agent.ephemeral_id.keyword": [
      "21fae111-5eec-4d88-b4ab-bb2f64b09bcb"
    ],
    "agent.name.keyword": [
      "DESKTOP-ONE7L9S"
    ],
    "winlog.event_data.CallerProcessId.keyword": [
      "0x15b4"
    ],
    "event.action": [
      "group-membership-enumerated"
    ],
    "@timestamp": [
      "2021-07-09T08:41:28.531Z"
    ],
    "winlog.channel": [
      "Security"
    ],
    "host.os.platform": [
      "windows"
    ],
    "winlog.event_data.TargetDomainName": [
      "DESKTOP-ONE7L9S"
    ],
    "winlog.opcode": [
      "Info"
    ],
    "agent.ephemeral_id": [
      "21fae111-5eec-4d88-b4ab-bb2f64b09bcb"
    ],
    "winlog.event_data.SubjectDomainName": [
      "WORKGROUP"
    ],
    "winlog.event_id.keyword": [
      "4798"
    ],
    "winlog.event_data.SubjectLogonId.keyword": [
      "0x3e7"
    ],
    "host.architecture.keyword": [
      "x86_64"
    ],
    "event.category.keyword": [
      "iam"
    ],
    "winlog.provider_guid": [
      "{54849625-5478-4994-a5ba-3e3b0328c30d}"
    ],
    "winlog.provider_name": [
      "Microsoft-Windows-Security-Auditing"
    ],
    "host.os.build.keyword": [
      "19043.1083"
    ],
    "winlog.event_data.SubjectUserName.keyword": [
      "DESKTOP-ONE7L9S$"
    ],
    "event.code.keyword": [
      "4798"
    ],
    "winlog.computer_name": [
      "DESKTOP-ONE7L9S"
    ],
    "user.id.keyword": [
      "S-1-5-18"
    ],
    "agent.hostname.keyword": [
      "DESKTOP-ONE7L9S"
    ],
    "ecs.version.keyword": [
      "1.9.0"
    ],
    "host.ip.keyword": [
      "fe80::144f:79e0:7649:9d5e",
      "169.254.157.94",
      "fe80::8d99:9241:be7e:3a48",
      "192.168.58.1",
      "fe80::a0c3:ece7:1118:2a78",
      "192.168.77.1",
      "2001:bb6:8266:fc58:785c:a917:280a:f49f",
      "2001:bb6:8266:fc58:b97a:b0c4:70d7:f130",
      "fe80::785c:a917:280a:f49f",
      "192.168.1.12"
    ],
    "winlog.keywords": [
      "Audit Success"
    ],
    "winlog.record_id": [
      162930
    ],
    "winlog.event_data.TargetUserName.keyword": [
      "ashwi"
    ],
    "winlog.keywords.keyword": [
      "Audit Success"
    ],
    "host.os.name": [
      "Windows 10 Home"
    ],
    "event.kind.keyword": [
      "event"
    ],
    "log.level": [
      "information"
    ],
    "event.action.keyword": [
      "group-membership-enumerated"
    ],
    "host.id.keyword": [
      "76e7c87e-b848-4546-a0ab-8edd66c1cd33"
    ],
    "host.name": [
      "DESKTOP-ONE7L9S"
    ],
    "host.os.version.keyword": [
      "10.0"
    ],
    "event.kind": [
      "event"
    ],
    "winlog.activity_id": [
      "{4dba53ce-73e5-0002-3654-ba4de573d701}"
    ],
    "winlog.event_data.TargetUserName": [
      "ashwi"
    ],
    "winlog.activity_id.keyword": [
      "{4dba53ce-73e5-0002-3654-ba4de573d701}"
    ],
    "winlog.event_data.SubjectDomainName.keyword": [
      "WORKGROUP"
    ],
    "@version.keyword": [
      "1"
    ],
    "ecs.version": [
      "1.9.0"
    ],
    "event.created": [
      "2021-07-09T08:41:30.212Z"
    ],
    "host.hostname.keyword": [
      "DESKTOP-ONE7L9S"
    ],
    "event.module.keyword": [
      "security"
    ],
    "agent.version": [
      "7.13.2"
    ],
    "winlog.event_data.SubjectUserSid.keyword": [
      "S-1-5-18"
    ],
    "host.os.family": [
      "windows"
    ],
    "user.domain.keyword": [
      "WORKGROUP"
    ],
    "log.level.keyword": [
      "information"
    ],
    "winlog.computer_name.keyword": [
      "DESKTOP-ONE7L9S"
    ],
    "host.os.build": [
      "19043.1083"
    ],
    "event.module": [
      "security"
    ],
    "host.os.kernel": [
      "10.0.19041.1083 (WinBuild.160101.0800)"
    ],
    "@version": [
      "1"
    ],
    "host.os.name.keyword": [
      "Windows 10 Home"
    ],
    "winlog.provider_guid.keyword": [
      "{54849625-5478-4994-a5ba-3e3b0328c30d}"
    ],
    "winlog.task": [
      "User Account Management"
    ],
    "winlog.task.keyword": [
      "User Account Management"
    ],
    "host.mac.keyword": [
      "8c:dc:d4:83:2a:e2",
      "00:50:56:c0:00:01",
      "00:50:56:c0:00:08",
      "f8:16:54:58:38:d3"
    ],
    "winlog.event_data.SubjectUserName": [
      "DESKTOP-ONE7L9S$"
    ],
    "message": [
      "A user's local group membership was enumerated.\n\nSubject:\n\tSecurity ID:\t\tS-1-5-18\n\tAccount Name:\t\tDESKTOP-ONE7L9S$\n\tAccount Domain:\t\tWORKGROUP\n\tLogon ID:\t\t0x3E7\n\nUser:\n\tSecurity ID:\t\tS-1-5-21-495902564-1190171751-3348208613-1001\n\tAccount Name:\t\tashwi\n\tAccount Domain:\t\tDESKTOP-ONE7L9S\n\nProcess Information:\n\tProcess ID:\t\t0x15b4\n\tProcess Name:\t\tC:\\Windows\\System32\\svchost.exe"
    ],
    "winlog.logon.id.keyword": [
      "0x3e7"
    ],
    "winlog.event_id": [
      "4798"
    ],
    "winlog.channel.keyword": [
      "Security"
    ],
    "host.os.family.keyword": [
      "windows"
    ],
    "host.os.type.keyword": [
      "windows"
    ],
    "host.os.platform.keyword": [
      "windows"
    ],
    "event.type": [
      "user",
      "info"
    ],
    "winlog.event_data.TargetSid.keyword": [
      "S-1-5-21-495902564-1190171751-3348208613-1001"
    ],
    "winlog.opcode.keyword": [
      "Info"
    ],
    "related.user.keyword": [
      "DESKTOP-ONE7L9S$",
      "ashwi"
    ],
    "user.name.keyword": [
      "DESKTOP-ONE7L9S$"
    ]
  }
}
9

There is no beat field in your document, so your conditional will return false, it is working as expected, you are comparing it with a field that does not exist so it will return false and will enter the else block, sending the log to the syslog- index.

But what you are doing is confusing, you said that all data were going to the index syslog-*, but the document you shared is from the index winlog-*.

Did you change anything? Can you explain exactly what the problem is and how are you ingestind the data? Also share your winlogbeat.yml configuration.

10

I changed the field name to [host][hostname] and it is working fine. Thank you :slight_smile:

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.