# Winlogbeat modules with logstash output

**URL:** <https://discuss.elastic.co/t/winlogbeat-modules-with-logstash-output/358464>\
**Category:** Beats\
**Tags:** winlogbeat\
**Created:** [April 30, 2024, 7:34am UTC](https://discuss.elastic.co/t/winlogbeat-modules-with-logstash-output/358464 "2024-04-30T07:34:58Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![s0p4L1n3](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/s0p4l1n3/32/128200_2.png) [@s0p4L1n3](https://discuss.elastic.co/u/s0p4L1n3)\
**Post date:** [April 30, 2024, 7:34am UTC](https://discuss.elastic.co/t/winlogbeat-modules-with-logstash-output/358464/1 "2024-04-30T07:34:58Z")

</div>

Hello,

I'm currently moving from Graylog solution to ELK stack solution for the security logs centralization.

On Graylog, I was using Winlogbeat 7.12.1 and now I'm using Winlogbeat 8.13.2.

I've successfully translated the 7.X winlogbeat configuration file to 8.X configuration especially for the security module BUT I'm stuck at some point or maybe it is a normal behaviour.

- Old Winlogbeat 7.X config:

```auto
- name: Security
     event_id: 4608, 4610, 4611, 4614, 4622, 4697, 4719, 4817, 4826, 4902, 4904, 4905, 4906, 4908, 4912
     ignore_older: 48h
     tags: [windowsos]
     processors:
      - script:
          lang: javascript
          id: security
          file: C:\Program Files\Graylog\sidecar\module\security\config\winlogbeat-security.js
      - drop_event.when:
           and:
             - equals.winlog.event_data.SubjectUserSid: 'S-1-5-18'

output.logstash:
  hosts: ["graylog.lab.lan:5044"]

```

- New config

```auto
- name: Security
     id: security-windows-os
     event_id: 4608, 4610, 4611, 4614, 4622, 4697, 4719, 4817, 4826, 4902, 4904, 4905, 4906, 4908, 4912
     ignore_older: 48h
     tags: [windowsos]
     processors:
      - drop_event.when:
           and:
             - equals.winlog.event_data.SubjectUserSid: 'S-1-5-18'

output.logstash:
  hosts: ["elk.lab.lan:5044"]

```

As the call of winlogbeat modules changed since 8.x, is there a way to says to winlogbeat agent to use the security and powershell modules ?

If I chose the output to be elastic, I can decide to use pipelines and security modules with the pipeline setting.

```auto
pipeline: "winlogbeat-%{[agent.version]}-routing"

```

But I want to send events to Logstash first, then writing GROK pattern to parse/add additionnal missing field OR to create fields based on certains conditions.

The documentation says

> All module processing is handled via Elasticsearch Ingest Node pipelines  
> If Winlogbeat is sending events to Logstash or another output you need to load the ingest pipelines with the `setup` command or manually.

When I run the winlogbeat setup --pipelines, I got this error below because it tries to setup through elasticsearch protocol not logstash.

```auto
Invalid version of beats protocol: 69

```

When the output is logstash, how I can verify that winlogbeat is using the modules ?

Thank you

---

<div class="post-metadata">

**Author:** ![NickFritts](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nickfritts/32/47189_2.png) [@NickFritts](https://discuss.elastic.co/u/NickFritts)\
**Post date:** [April 30, 2024, 9:44am UTC](https://discuss.elastic.co/t/winlogbeat-modules-with-logstash-output/358464/2 "2024-04-30T09:44:48Z")

</div>

Howdy @s0p4L1n3 ,

Have you seen this documentation page: [Working with Winlogbeat Modules | Logstash Reference [8.13] | Elastic](https://www.elastic.co/guide/en/logstash/8.13/winlogbeat-modules.html#use-winlogbeat-ingest-pipelines) ?

Specifically the note after it shows the `winlogbeat setup` command:

> A connection to Elasticsearch is required for this setup step because Winlogbeat needs to load the ingest pipelines into Elasticsearch. If necessary, you can temporarily disable your configured output and enable the Elasticsearch output before running the command.

---

<div class="post-metadata">

**Author:** ![s0p4L1n3](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/s0p4l1n3/32/128200_2.png) [@s0p4L1n3](https://discuss.elastic.co/u/s0p4L1n3)\
**Post date:** [April 30, 2024, 9:54am UTC](https://discuss.elastic.co/t/winlogbeat-modules-with-logstash-output/358464/3 "2024-04-30T09:54:11Z")

</div>

Yes I read it. I did at first set up Elastic output so I assume it already ingested the pipelines.

The [documentation](https://www.elastic.co/guide/en/beats/winlogbeat/current/winlogbeat-modules.html) also says:

> Winlogbeat’s default config file contains the option to send all events to the routing pipeline. If you remove this option then the module processing will not be applied.  
> `output.elasticsearch.pipeline: winlogbeat-%{[agent.version]}-routing`

There is no `winlogbeat-%{[agent.version]}-routing` for Logstash, does winlogbeat process the modules when logstash is used as output ?

---

<div class="post-metadata">

**Author:** ![s0p4L1n3](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/s0p4l1n3/32/128200_2.png) [@s0p4L1n3](https://discuss.elastic.co/u/s0p4L1n3)\
**Post date:** [April 30, 2024, 10:03am UTC](https://discuss.elastic.co/t/winlogbeat-modules-with-logstash-output/358464/4 "2024-04-30T10:03:51Z")

</div>

And If I change my logstash conf from:

```auto
input {
        beats {
                port => 5044
        }
}

## Add your filters / logstash plugins configuration here

output {
    elasticsearch {
      hosts => "elasticsearch:9200"
      user => "logstash_internal"
      password => "${LOGSTASH_INTERNAL_PASSWORD}"
    }
  } 

```

to:

```auto
input {
        beats {
                port => 5044
        }
}

## Add your filters / logstash plugins configuration here

output {
  if [@metadata][pipeline] {
    elasticsearch {
      hosts => "elasticsearch:9200"
      manage_template => false
      index => "%{[@metadata][beat]}-%{[@metadata][version]}"
      action => "create"
      pipeline => "%{[@metadata][pipeline]}"
      user => "logstash_internal"
      password => "${LOGSTASH_INTERNAL_PASSWORD}"
    }
  } else {
    elasticsearch {
      hosts => "elasticsearch:9200"
      manage_template => false
      index => "%{[@metadata][beat]}-%{[@metadata][version]}"
      action => "create"
      user => "logstash_internal"
      password = "${LOGSTASH_INTERNAL_PASSWORD}"
    }
  }
}

```

I got some errors:

```auto
 Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [\\t\\r\\n], \"#\", \"=>\" at line 36, column 16 (byte 758) after output {\n if [@metadata][pipeline] {\n elasticsearch {\n hosts => \"elasticsearch:9200\"\n manage_template => false\n index => \"%{[@metadata][beat]}-%{[@metadata][version]}\" \n action => \"create\" \n pipeline => \"%{[@metadata][pipeline]}\" \n user => \"logstash_internal\"\n password => \"${LOGSTASH_INTERNAL_PASSWORD}\"\n }\n } else {\n elasticsearch {\n hosts => \"elasticsearch:9200\"\n manage_template => false\n index => \"%{[@metadata][beat]}-%{[@metadata][version]}\" \n action => \"create\"\n user => \"logstash_internal\"\n password ", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:32:in `compile_imperative'", "org/logstash/execution/AbstractPipelineExt.java:239:in `initialize'", "org/logstash/execution/AbstractPipelineExt.java:173:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:48:in `initialize'", "org/jruby/RubyClass.java:931:in `new'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:49:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:386:in `block in converge_state'"]}

```

---

<div class="post-metadata">

**Author:** ![s0p4L1n3](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/s0p4l1n3/32/128200_2.png) [@s0p4L1n3](https://discuss.elastic.co/u/s0p4L1n3)\
**Post date:** [April 30, 2024, 11:04am UTC](https://discuss.elastic.co/t/winlogbeat-modules-with-logstash-output/358464/5 "2024-04-30T11:04:44Z")

</div>

A character was missing for my logstash conf, but now I have another error:

```auto
Retrying failed action {:status=>403, :action=>["create", {:_id=>nil, :_index=>"winlogbeat-8.13.2", :routing=>nil}, {"tags"=>["filesystem", "beats_input_codec_plain_applied"], "agent"=>{"id"=>"7d7b46ea-dbef-4ea3-ac8c-3e1ca717e983", "ephemeral_id"=>"d3a47579-53fc-4c13-ad98-3522aada7d3e", "version"=>"8.13.2", 

```

It look like it try to send the routing module but fails

```auto
logstash | [2024-04-30T11:03:52,203][INFO][logstash.outputs.elasticsearch][main][922faaeb295f383636b4b7c407971a1af4b56664c571c61ae2fc4d513c4486c8] Retrying individual bulk actions that failed or were rejected by the previous bulk request {:count=>125}

```

I have the reason:

```auto
reason"=>"action [indices:data/write/bulk[s]] is unauthorized for user [logstash_internal] with effective roles [logstash_writer] on indices [winlogbeat-8.13.2], this action is granted by the index privileges [create_doc,create,delete,index,write,all]"}}

```

I will try to fix the permissions and post the outcome.

---

<div class="post-metadata">

**Author:** ![s0p4L1n3](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/s0p4l1n3/32/128200_2.png) [@s0p4L1n3](https://discuss.elastic.co/u/s0p4L1n3)\
**Post date:** [April 30, 2024, 11:14am UTC](https://discuss.elastic.co/t/winlogbeat-modules-with-logstash-output/358464/6 "2024-04-30T11:14:45Z")

</div>

I fixed my problem that was simple in the end, I just didn't read my logstash logs till the end of the line where the problem was displayed.

I've added the index privileges to the logstash\_writer role which is used by my logstash user.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/2/3/23ac8894a575972a230c437633534f9b9f4a3ee6.png)