# Winlogbeat via Logstash into Elasticsearch

**URL:** <https://discuss.elastic.co/t/winlogbeat-via-logstash-into-elasticsearch/90081>\
**Category:** Logstash\
**Created:** [June 20, 2017, 10:38am UTC](https://discuss.elastic.co/t/winlogbeat-via-logstash-into-elasticsearch/90081 "2017-06-20T10:38:05Z")\
**Posts on this page:** 19\
**Page:** 1

<div class="post-metadata">

**Author:** ![hispeed](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hispeed/32/16232_2.png) [@hispeed](https://discuss.elastic.co/u/hispeed)\
**Post date:** [June 20, 2017, 10:38am UTC](https://discuss.elastic.co/t/winlogbeat-via-logstash-into-elasticsearch/90081/1 "2017-06-20T10:38:05Z")

</div>

Hi,

I have a on a Windows machine winlogbeat and it's working fine so far. At the moment all logs are going into the same index in elasticsearch. Yes i have searched and found topics with that question but i was unable to solve it.  
I want that Logstash is saving winlogbeat documents into a different index.

I have this at the moment:

> ```
> output {
> # file {
> # path => "/tmp/out.log"
> # }
> elasticsearch { hosts => ["localhost:9200"]
> hosts => "localhost:9200"
> index => "logstash-%{+YYYY.MM.dd}"
> }
> index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
> document_type => "%{[@metadata][type]}"
> }
> # if [@metadata][cthostmeta] == "ELKSTACK" {
> # file {
> # codec => rubydebug { metadata => true}
> # path => "/tmp/cthost_out.log"
> # }
> # }
> }
> 
> ```

Thanks for your help.

---

<div class="post-metadata">

**Author:** ![jsvd](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsvd/32/6203_2.png) [@jsvd](https://discuss.elastic.co/u/jsvd)\
**Post date:** [June 20, 2017, 11:09am UTC](https://discuss.elastic.co/t/winlogbeat-via-logstash-into-elasticsearch/90081/2 "2017-06-20T11:09:19Z")

</div>

what is your input section in logstash like? you can tag the beats input events with a `winlogbeat` tag and the use that in the output section with a conditional

---

<div class="post-metadata">

**Author:** ![hispeed](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hispeed/32/16232_2.png) [@hispeed](https://discuss.elastic.co/u/hispeed)\
**Post date:** [June 20, 2017, 3:51pm UTC](https://discuss.elastic.co/t/winlogbeat-via-logstash-into-elasticsearch/90081/3 "2017-06-20T15:51:43Z")

</div>

Hi jsvd,

That sounds great. Input in Logstash looks like that:

```
syslog {
    port => 5514
    type => "syslog"
}
udp {
    type => "pfsense"
    port => 5140
}
beats {
    type => "log"
    port => 5044
}
udp {
    type => "syslog"
    port => 5515

}

```

Can you give an example how i have to do that?

---

<div class="post-metadata">

**Author:** ![jsvd](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsvd/32/6203_2.png) [@jsvd](https://discuss.elastic.co/u/jsvd)\
**Post date:** [June 20, 2017, 4:08pm UTC](https://discuss.elastic.co/t/winlogbeat-via-logstash-into-elasticsearch/90081/4 "2017-06-20T16:08:10Z")

</div>

looking at the winlogbeat docs, it seems that if you don't set the type , then it will be filled out by winlogbeat with either wineventlog or eventlogging: [https://www.elastic.co/guide/en/beats/winlogbeat/current/exported-fields-common.html](https://www.elastic.co/guide/en/beats/winlogbeat/current/exported-fields-common.html)

---

<div class="post-metadata">

**Author:** ![hispeed](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hispeed/32/16232_2.png) [@hispeed](https://discuss.elastic.co/u/hispeed)\
**Post date:** [June 20, 2017, 4:19pm UTC](https://discuss.elastic.co/t/winlogbeat-via-logstash-into-elasticsearch/90081/5 "2017-06-20T16:19:12Z")

</div>

Ok, but this doesn't help me to create different indexes?  
I have a Windows Server 2016 which is sending the events.

```
#----------------------------- Logstash output --------------------------------
output.logstash:
  # The Logstash hosts
  hosts: ["192.168.0.XX:5044"]
```

---

<div class="post-metadata">

**Author:** ![jsvd](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsvd/32/6203_2.png) [@jsvd](https://discuss.elastic.co/u/jsvd)\
**Post date:** [June 20, 2017, 4:20pm UTC](https://discuss.elastic.co/t/winlogbeat-via-logstash-into-elasticsearch/90081/6 "2017-06-20T16:20:51Z")

</div>

```auto
output {
  if [type] == "winlogbeat" {
    elasticsearch { index => "logstash-winlogbeat-%{YYYY.mm.dd}"
  } else {
    elasticsearch { } # normal flow
}

```

---

<div class="post-metadata">

**Author:** ![hispeed](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hispeed/32/16232_2.png) [@hispeed](https://discuss.elastic.co/u/hispeed)\
**Post date:** [June 20, 2017, 4:58pm UTC](https://discuss.elastic.co/t/winlogbeat-via-logstash-into-elasticsearch/90081/7 "2017-06-20T16:58:11Z")

</div>

I get the following error.

```
[2017-06-20T18:55:07,147][ERROR][logstash.agent] Cannot create pipeline {:reason=>"Expected one of #, => at line 33, column 17 (byte 534) after output {\n# file {\n# path => \"/tmp/out.log\"\n# }\n if [type] == \"winlogbeat\" {\n elasticsearch { index => \"logstash-winlogbeat-%{YYYY.mm.dd}\"\n } else {\n elasticsearch "}

> output {
> # file {
> # path => "/tmp/out.log"
> # }
> if [type] == "winlogbeat" {
> elasticsearch { index => "logstash-winlogbeat-%{YYYY.mm.dd}"
> } else {
> elasticsearch { hosts => ["localhost:9200"]
> hosts => "localhost:9200"
> index => "logstash-%{+YYYY.MM.dd}"
> }
> # }
> # if [@metadata][cthostmeta] == "ELKSTACK" {
> # file {
> # codec => rubydebug { metadata => true}
> # path => "/tmp/cthost_out.log"
> # }
> # }
> }

```

Somwhere I have a mistake... i have tried different variants...

---

<div class="post-metadata">

**Author:** ![jsvd](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsvd/32/6203_2.png) [@jsvd](https://discuss.elastic.co/u/jsvd)\
**Post date:** [June 20, 2017, 5:18pm UTC](https://discuss.elastic.co/t/winlogbeat-via-logstash-into-elasticsearch/90081/8 "2017-06-20T17:18:52Z")

</div>

> [@hispeed](#):
>
> \> elasticsearch { index =\> "logstash-winlogbeat-%{YYYY.mm.dd}"

both elasticsearch plugins are missing a }

> ```
> elasticsearch { index => "logstash-winlogbeat-%{YYYY.mm.dd}"
> 
> ```

should be

> ```
> elasticsearch { index => "logstash-winlogbeat-%{YYYY.mm.dd}" }
> 
> ```

also  
elasticsearch { hosts =\> ["localhost:9200"]

> ```
> hosts => "localhost:9200"
> index => "logstash-%{+YYYY.MM.dd}"
> 
> ```
> 
> }

has hosts twice

---

<div class="post-metadata">

**Author:** ![hispeed](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hispeed/32/16232_2.png) [@hispeed](https://discuss.elastic.co/u/hispeed)\
**Post date:** [June 20, 2017, 5:48pm UTC](https://discuss.elastic.co/t/winlogbeat-via-logstash-into-elasticsearch/90081/9 "2017-06-20T17:48:26Z")

</div>

Thanks now I'm back online.

Hmm but it didn't make a new indices now for winlogbeat. If i do: GET /\_cat/indices?v in Kibana i only see my logstash indices.  
They are comming now in like that:

```
t _index logstash-2017.06.20
 # _score - 
 t _type wineventlog

```

I probably have this problem with one field. How should i go to fix that?  
Mapping conflict field:  
event\_data.param1 conflict

Update:  
I can't create a new index pattern in Kibana with: logstash-winlogbeat-\*

---

<div class="post-metadata">

**Author:** ![jsvd](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsvd/32/6203_2.png) [@jsvd](https://discuss.elastic.co/u/jsvd)\
**Post date:** [June 21, 2017, 8:56am UTC](https://discuss.elastic.co/t/winlogbeat-via-logstash-into-elasticsearch/90081/10 "2017-06-21T08:56:20Z")

</div>

this seem to be problems with how elasticsearch is configured and the already existing data in indices.  
can you post error messages somewhere?

also, you can confirm which events are going where by putting a `stdout { codec => rubydebug } }` before a `elasticsearch { .. }` block, that will debug the events going to that elasticsearch

---

<div class="post-metadata">

**Author:** ![hispeed](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hispeed/32/16232_2.png) [@hispeed](https://discuss.elastic.co/u/hispeed)\
**Post date:** [June 21, 2017, 12:01pm UTC](https://discuss.elastic.co/t/winlogbeat-via-logstash-into-elasticsearch/90081/11 "2017-06-21T12:01:06Z")

</div>

I have putted that before the elasticsearch.

Sorry i'm new to the whole elasticsearch. Which logs do you mean?

I found also the following:

> [2017-06-21T13:51:17,545][INFO][logstash.inputs.beats] Beats inputs: Starting input listener {:address=\>"0.0.0.0:5044"}

EDIT:  
Where can I now see the result from stdout?

---

<div class="post-metadata">

**Author:** ![jsvd](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsvd/32/6203_2.png) [@jsvd](https://discuss.elastic.co/u/jsvd)\
**Post date:** [June 21, 2017, 12:43pm UTC](https://discuss.elastic.co/t/winlogbeat-via-logstash-into-elasticsearch/90081/12 "2017-06-21T12:43:48Z")

</div>

you can place the stdout block in different places to see the events that reach that section. example:

```auto
input {
  beats { }
  tcp {}
}
output {
  stdout { codec => rubydebug } # option 1: here all events will be logged
  if [type] == "winlogbeat" {
    stdout { codec => rubydebug } # option 2: here only all events with that tag will be logged
    elasticsearch { index => "logstash-winlogbeat-%{YYYY.mm.dd}" }
  } else {
    elasticsearch { index => "logstash-winlogbeat-%{YYYY.mm.dd}" }
  }
}

```

If you don't see events being printed to stdout it either means data is not arriving in the inputs, or your conditional is not being met (e.g. type isn't really winlogbeat)

---

<div class="post-metadata">

**Author:** ![hispeed](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hispeed/32/16232_2.png) [@hispeed](https://discuss.elastic.co/u/hispeed)\
**Post date:** [June 21, 2017, 7:32pm UTC](https://discuss.elastic.co/t/winlogbeat-via-logstash-into-elasticsearch/90081/13 "2017-06-21T19:32:54Z")

</div>

I understand that. My problem is that I don't know now **where can I look into** the output **stdout**?

I have a installation on CentOS 7 based on this tutorial: [https://www.howtoforge.com/tutorial/how-to-install-elastic-stack-on-centos-7/](https://www.howtoforge.com/tutorial/how-to-install-elastic-stack-on-centos-7/)

Probably I found also a problem. I have done that with the certificate so maybe there is now a problem.

---

<div class="post-metadata">

**Author:** ![jsvd](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsvd/32/6203_2.png) [@jsvd](https://discuss.elastic.co/u/jsvd)\
**Post date:** [June 22, 2017, 7:18am UTC](https://discuss.elastic.co/t/winlogbeat-via-logstash-into-elasticsearch/90081/14 "2017-06-22T07:18:58Z")

</div>

ah I didn't realise you were using the rpm packages. then you can replace:

`stdout { codec => rubydebug }` with something like  
`file { path => '/tmp/logstash.output.debug.txt' codec => rubydebug }`

---

<div class="post-metadata">

**Author:** ![hispeed](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hispeed/32/16232_2.png) [@hispeed](https://discuss.elastic.co/u/hispeed)\
**Post date:** [June 22, 2017, 8:21pm UTC](https://discuss.elastic.co/t/winlogbeat-via-logstash-into-elasticsearch/90081/15 "2017-06-22T20:21:00Z")

</div>

Ahhh so now i got it.

You wanted to see that:

```
"record_number" => "50939",
   "event_data" => {
       "Workstation" => "XXXXXLP",
            "Status" => "0x0",
       "PackageName" => "MICROSOFT_AUTHENTICATION_PACKAGE_V1_0",
    "TargetUserName" => "XXXXX"
},
      "message" => "Es wurde versucht, die Anmeldeinformationen für ein Konto zu überprüfen.\n\nAuthentifizierungspaket:\tMICROSOFT_AUTHENTICATION_PACKAGE_V1_0\nAnmeldekonto:\tMarc.Walter\nArbeitsstation:\tHISPEEDLP\nFehlercode:\t0x0",
         "type" => "wineventlog",
       "opcode" => "Info",
         "tags" => [
    [0] "beats_input_codec_plain_applied"
],
    "thread_id" => 180,
   "@timestamp" => 2017-06-22T20:08:34.398Z,
         "task" => "Überprüfung der Anmeldeinformationen",
     "event_id" => 4776,
"provider_guid" => "{54849625-5478-4994-A5BA-3E3B0328C30D}",
     "@version" => "1",
         "beat" => {
    "hostname" => "XXXXXX",
        "name" => "XXXXXX",
     "version" => "5.4.0"
},
  "activity_id" => "{13ED4890-D0D8-0003-9C48-ED13D8D0D201}",
         "host" => "XXXXXXXX",
  "source_name" => "Microsoft-Windows-Security-Auditing"
}
{
   "process_id" => 760,
"computer_name" => "XXXXXXX",
     "keywords" => [
    [0] "Überwachung erfolgreich"
],
        "level" => "Informationen",
     "log_name" => "Security",
    "@metadata" => {
    "beat" => "winlogbeat",
    "type" => "wineventlog"
},
"record_number" => "50940",
```

---

<div class="post-metadata">

**Author:** ![jsvd](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsvd/32/6203_2.png) [@jsvd](https://discuss.elastic.co/u/jsvd)\
**Post date:** [June 22, 2017, 10:21pm UTC](https://discuss.elastic.co/t/winlogbeat-via-logstash-into-elasticsearch/90081/16 "2017-06-22T22:21:27Z")

</div>

ok so we confirmed that data from winlogbeat has type == "wineventlog", now you can write the proper conditional, like:

```auto
output {
  if [type] == "wineventlog" {
    elasticsearch { index => "logstash-eventlog-%{+YYYY.MM.dd}" }
  } else {
    elasticsearch { index => "logstash-%{+YYYY.MM.dd}" }
  }
}

```

---

<div class="post-metadata">

**Author:** ![hispeed](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hispeed/32/16232_2.png) [@hispeed](https://discuss.elastic.co/u/hispeed)\
**Post date:** [June 23, 2017, 10:10am UTC](https://discuss.elastic.co/t/winlogbeat-via-logstash-into-elasticsearch/90081/17 "2017-06-23T10:10:20Z")

</div>

I have added it. It looks now like that:

I can't create: logstash-winlogbeat-\* the index pattern in Kibana.

```
output {
# file {
# path => "/tmp/out.log"
# }
 if [type] == "wineventlog" {
    elasticsearch { index => "logstash-winlogbeat-%{YYYY.mm.dd}" }
  } else {
  elasticsearch {
     hosts => "localhost:9200"
     index => "logstash-%{+YYYY.MM.dd}"
   }
 }
# if [@metadata][cthostmeta] == "ELKSTACK" {
# file {
# codec => rubydebug { metadata => true}
# path => "/tmp/logstash_out.log"
# }
# }
}
```

---

<div class="post-metadata">

**Author:** ![hispeed](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hispeed/32/16232_2.png) [@hispeed](https://discuss.elastic.co/u/hispeed)\
**Post date:** [June 24, 2017, 5:40am UTC](https://discuss.elastic.co/t/winlogbeat-via-logstash-into-elasticsearch/90081/18 "2017-06-24T05:40:32Z")

</div>

Kibana shows now today as following:

 ![](https://us1.discourse-cdn.com/elastic/original/3X/5/6/5634f5568e4ac19cefdcad6eb048f395a92ab2f8.png)

Unfortunately I can't create the index. Because I already have a logstash-\* Index?

It also shows me the eventlog indices in the searches panel: No matching indices found: [index\_not\_found\_exception] no such index, with { resource.type="index\_or\_alias" & resource.id="logstash-eventlog-" & index\_uuid="na" & index="logstash-eventlog-" }  
But i recieve there this message. When I look into the indices via Kibana I can't see a Indices like: logstash-eventlog-\*

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 22, 2017, 5:40am UTC](https://discuss.elastic.co/t/winlogbeat-via-logstash-into-elasticsearch/90081/19 "2017-07-22T05:40:37Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.