# Winlogbeat logs sent through logstash aren't parsed correctly

**URL:** <https://discuss.elastic.co/t/winlogbeat-logs-sent-through-logstash-arent-parsed-correctly/338076>\
**Category:** Beats\
**Tags:** winlogbeat\
**Created:** [July 11, 2023, 8:32am UTC](https://discuss.elastic.co/t/winlogbeat-logs-sent-through-logstash-arent-parsed-correctly/338076 "2023-07-11T08:32:42Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![dosterberg](https://avatars.discourse-cdn.com/v4/letter/d/9f8e36/32.png) [@dosterberg](https://discuss.elastic.co/u/dosterberg)\
**Post date:** [July 11, 2023, 8:32am UTC](https://discuss.elastic.co/t/winlogbeat-logs-sent-through-logstash-arent-parsed-correctly/338076/1 "2023-07-11T08:32:42Z")

</div>

Hi everyone! I'm new to ELK and have been enjoying very much working with it so far. I am currently evaluating ELK with Elastic Security as a SIEM in a test environment. I have tried sending both data from filebeat with the system module, and auditbeat with all its modules through a server with logstash that sends the logs to Elastic Search/Kibana. All of these logs are parsed perfectly and I get all the fields needed for Elastic Security functionality such as event.category, and thus the hosts show up in Elastic Security with "uncommon processes", "authentications" etc.

However, I am also sending logs from winlogbeat to the same logstash input, and while these events get a lot of the correct fields in Kibana, vital fields for Elastic Security such as event.category aren't added, and I either get a bunch of data in the event.original field or ignored\_field\_values. Because of this "authentications", "uncommon processes" and likewise don't show for the hosts in Elastic Security. When I send the logs from Winlogbeat directly to Elasticsearch, the event.category field shows up correctly. My guess is that Logstash isn't using the correct pipeline when parsing the logs and sending them to Elasticsearch, but as far as I can see I have done it according to the documentation which explains the same method as for the other beats. If someone could help me with what I might be doing wrong it would be greatly appreciated.

Vital information from a winlogbeat.yml file (I have a lot more events in the processors and didn't add them all here to keep the summary shorter, I have also tried not filtering events at all):

```auto
winlogbeat.event_logs:

  - name: Security
    processors:
      - drop_event.when.not.or:
        - equals.winlog.event_id: "4608"
        - equals.winlog.event_id: "4609"
        - equals.winlog.event_id: "4610"

  - name: Microsoft-Windows-Sysmon/Operational
    processors:
      - drop_event.when.not.or:
        - equals.winlog.event_id: "4608"
        - equals.winlog.event_id: "4609"
        - equals.winlog.event_id: "4610"

  - name: Windows PowerShell
    processors:
      - drop_event.when.not.or:
        - equals.winlog.event_id: "4608"
        - equals.winlog.event_id: "4609"
        - equals.winlog.event_id: "4610"

output.logstash:

  hosts: ["x.x.x.x:5044"]

processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~

```

Vital information from auditbeat.yml since this is working (modules are enabled outside of the file):

```auto
output.logstash:
  hosts: ["100.72.20.38:5044"]

processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~
  - add_docker_metadata: ~

```

Logstash server running on Rocky Linux 9, file /etc/logstash/conf.d/beats.conf that's used for all beats inputs (I will configure this to use more security best practices such as https for the input later but I'm just trying to get the basic functionality to work right now):

```auto
input {
    beats {
        port => 5044
      }
}
output {
  if [@metadata][pipeline] {
    elasticsearch {
      hosts => "https://x.x.x.x:9200"
      manage_template => false
      index => "%{[@metadata][beat]}-%{[@metadata][version]}"
      action => "create"
      pipeline => "%{[@metadata][pipeline]}"
      user => "xxx"
      password => "xxx"
      cacert => 'xxxt'
    }
  } else {
    elasticsearch {
      hosts => "https://x.x.x.x:9200"
      manage_template => false
      index => "%{[@metadata][beat]}-%{[@metadata][version]}"
      action => "create"
      user => "xxx"
      password => "xxx"
      cacert => 'xxx'
    }
  }
}

```

When I tried adding pipeline =\> "%{[@metadata][pipeline]}" to the "else" block I get the following error:

_:response=\>{"create"=\>{"\_index"=\>"winlogbeat-8.8.2", "\_id"=\>nil, "status"=\>400, "error"=\>{"type"=\>"illegal\_argument\_exception", "reason"=\>"pipeline with id [%{[@metadata][pipeline]}] does not exist"}}}}_

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [July 11, 2023, 11:42am UTC](https://discuss.elastic.co/t/winlogbeat-logs-sent-through-logstash-arent-parsed-correctly/338076/2 "2023-07-11T11:42:44Z")

</div>

Assuming you have the pipelines installed in Elasticsearch, can you try using this general logic in the LS config

```auto
  else if [@metadata][beat] == "winlogbeat" {
    elasticsearch {
      ...
      pipeline => "%{[@metadata][beat]}-%{[@metadata][version]}-routing"

```

---

<div class="post-metadata">

**Author:** ![dosterberg](https://avatars.discourse-cdn.com/v4/letter/d/9f8e36/32.png) [@dosterberg](https://discuss.elastic.co/u/dosterberg)\
**Post date:** [July 11, 2023, 12:12pm UTC](https://discuss.elastic.co/t/winlogbeat-logs-sent-through-logstash-arent-parsed-correctly/338076/3 "2023-07-11T12:12:20Z")

</div>

Hi Andrew,

That actually seems to have solved the problem, thank you very much for the quick help!

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [July 11, 2023, 12:51pm UTC](https://discuss.elastic.co/t/winlogbeat-logs-sent-through-logstash-arent-parsed-correctly/338076/4 "2023-07-11T12:51:00Z")

</div>

I created an issue to improve the documentation for Winlogbeat modules.

> <https://github.com/elastic/beats/issues/36037>
>
> The documentation for Winlogbeat modules should show an example of how to config…ure Logstash to route data to the "routing" pipeline. I have seen a number of questions relating to this.
> 
> https://www.elastic.co/guide/en/beats/winlogbeat/current/winlogbeat-modules.html
> 
> https://discuss.elastic.co/t/winlogbeat-logs-sent-through-logstash-arent-parsed-correctly/338076

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 8, 2023, 2:51pm UTC](https://discuss.elastic.co/t/winlogbeat-logs-sent-through-logstash-arent-parsed-correctly/338076/5 "2023-08-08T14:51:28Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.