# Unable to index winlogbeat using logstash

**URL:** https://discuss.elastic.co/t/unable-to-index-winlogbeat-using-logstash/316434
**Category:** Logstash
**Created:** [October 12, 2022, 12:02pm UTC](https://discuss.elastic.co/t/unable-to-index-winlogbeat-using-logstash/316434 "2022-10-12T12:02:07Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![bamfery](https://avatars.discourse-cdn.com/v4/letter/b/ba9def/32.png) [@bamfery](https://discuss.elastic.co/u/bamfery)
#### Post date: [October 12, 2022, 12:02pm UTC](https://discuss.elastic.co/t/unable-to-index-winlogbeat-using-logstash/316434/1 "2022-10-12T12:02:07Z")

</div>

I am trying to set up winlogbeat to ship data to a logstash server, which further forwards to another logstash server and then to elasticsearch. I am seeing that when I add index =\> "%{[@metadata][beat]}-%{+YYYY.MM.dd}" to the elasticsearch output plugin that the metadata fields are not accessible. I am guessing that the problem is that Logstash removes the metadata fields, so i found this troubleshooting tip, however I don't understand how to apply it. [https://www.elastic.co/guide/en/beats/winlogbeat/current/metadata-missing.html](https://www.elastic.co/guide/en/beats/winlogbeat/current/metadata-missing.html)  
Winlogbeat configuration:

```auto
winlogbeat.event_logs:
  - name: Application
    ignore_older: 72h
  - name: System
  - name: Security
  - name: Windows PowerShell
  - name: Microsoft-Windows-PowerShell/Operational
  - name: Microsoft-Windows-Windows Defender/Operational
  - name: Microsoft-Windows-Windows Firewall With Advanced Security/Firewall
  - name: ForwardedEvents
    tags: [forwarded]

setup.template.enabled: false

output.logstash:
  hosts: ["192.168.243.146:5044"]
  index: winlogbeat

#output.elasticsearch:
# hosts: ["https://192.168.243.143:9200"]
# username: "elastic"
# password: "l1vEHXlK_zcwB*94rF4q"
# index: winlogbeat
# ssl.certificate_authorities: ['C:\Program Files\winlogbeat-8.4.1-windows-x86_64\http_ca.crt']

processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_cloud_metadata: ~

```

Logstash 1 configuration (Logstash to Logstash):

```auto
input {
  beats {
    port => 5044

  }
}

output {
  lumberjack {
    hosts => ["192.168.243.145"]
    port => 5044
    ssl_certificate => 'C:\Users\Administrator\Desktop\logstash-8.4.2-windows-x86_64\logstash-8.4.2\config\certs\lumberjack.cert'
    codec => json
  }
stdout { codec => rubydebug { metadata => true } }
}

```

Logstash 2 configuration (Logstash to Elasticsearch):

```auto
input {
  beats {
    port => 5044
    ssl => true
    codec => json
    ssl_certificate => 'C:\Users\Administrator\Desktop\logstash-8.4.2-windows-x86_64\logstash-8.4.2\config\certs\lumberjack.cert'
    ssl_key => 'C:\Users\Administrator\Desktop\logstash-8.4.2-windows-x86_64\logstash-8.4.2\config\certs\lumberjack.key'
 }
}

output {
    elasticsearch {
      hosts => ["https://192.168.243.143:9200"]
      ssl => "true"
      cacert => 'C:\Users\Administrator\Desktop\logstash-8.4.2-windows-x86_64\logstash-8.4.2\config\certs\http_ca.crt'
      user => "elastic"
      password => "l1vEHXlK_zcwB*94rF4q"
      #pipeline => "%{[@metadata][pipeline]}"
      manage_template => false
      index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
      #document_type => "%{[@metadata][type]}"
    }
     stdout { codec => rubydebug { metadata => true } }
  }

```

Index that is created in Kibana:  
 ![kibana](https://us1.discourse-cdn.com/elastic/original/3X/5/3/5307728d132ce248c4ba4e7d80d4451fef5a2cb5.png)

---

<div class="post-metadata">

### Author: ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [October 12, 2022, 1:53pm UTC](https://discuss.elastic.co/t/unable-to-index-winlogbeat-using-logstash/316434/2 "2022-10-12T13:53:37Z")

</div>

As explained in the documentation you need to create a new field in your first logstash that will have the value of the `@metadata` field you want to use.

> To preserve `@metadata` fields, use the Logstash mutate filter with the rename setting to rename the fields to non-internal fields.

You need something like this in your first logstash

```auto
filter {
    mutate {
        rename => {
            "[@metadata][beat]" => "[index_name]"
        }
    }
}

```

Then in your second logstash you will use:

```auto
index => "%{[index_name]}-%{+YYYY.MM.dd}"

```

If you want to use other `@metadata` fields you will also need to rename them to new fields.

---

<div class="post-metadata">

### Author: ![bamfery](https://avatars.discourse-cdn.com/v4/letter/b/ba9def/32.png) [@bamfery](https://discuss.elastic.co/u/bamfery)
#### Post date: [October 13, 2022, 6:53am UTC](https://discuss.elastic.co/t/unable-to-index-winlogbeat-using-logstash/316434/3 "2022-10-13T06:53:09Z")

</div>

Thank you that worked out nicely! What else do I need to do in my config in order to get the exact same setup as if I was to connect winlogbeat directly to elasticsearch? I've loaded the template and pipelines into elasticsearch so far, but I'm not sure why it is not working. Again, preciate the help 🙂

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 10, 2022, 6:53am UTC](https://discuss.elastic.co/t/unable-to-index-winlogbeat-using-logstash/316434/4 "2022-11-10T06:53:21Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
