# Kafka output doesnt use winlogbeat template

**URL:** <https://discuss.elastic.co/t/kafka-output-doesnt-use-winlogbeat-template/218310>\
**Category:** Logstash\
**Tags:** docker\
**Created:** [February 7, 2020, 10:02am UTC](https://discuss.elastic.co/t/kafka-output-doesnt-use-winlogbeat-template/218310 "2020-02-07T10:02:31Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![niki](https://avatars.discourse-cdn.com/v4/letter/n/c57346/32.png) [@niki](https://discuss.elastic.co/u/niki)\
**Post date:** [February 7, 2020, 10:02am UTC](https://discuss.elastic.co/t/kafka-output-doesnt-use-winlogbeat-template/218310/1 "2020-02-07T10:02:31Z")

</div>

I am trying to set up an elk stack with apache kafka as log broker. so here is the situation:

1. elasticsearch, logstash, zookeeper, kafka and kibana installed with docker compose and each one in different container(you can see the docker-compose.yml below)

2. kafka listens on port 9092 and recives windows event logs from winlogbeat.

3. logstash is configured as a consumer of winlogbeat topic in kafka.(you can see pipeline.conf below)

4. I already put winlogbeat template into elasticsearch using curl command.  
the problemn is that data wont use winlogbeat template and entire event message will appear only in message field.(ill share screenshots below)  
how ever if I use elasticsearch.output and send data straight into elasticsearch everythings are fine.  
and ofcourse data from filebeat with correct filters are fine and I still cant figure it out why winlogbeat data wont use index template.  
here is docker-compose.yml file:

and here is pipeline.conf belongs to logstash container:

```
input {
  kafka {
    bootstrap_servers => "kafka01:9092"
    topics => ["winlogbeat","filebeat"]
    decorate_events => true
  }
}
output {
  elasticsearch {
    hosts => ["elasticsearch:9200"]
    index => "winlogbeat-7.5.2"
    manage_template => true
    template => "/tmp/winlogbeat.template.json"
    template_overwrite => "true"
    codec => json
  }
}

```

and here is the screenshot of kibana shows that pipeline is working but cant use the winlogbeat template.

 ![Capture](https://us1.discourse-cdn.com/elastic/original/3X/a/b/abacd36d198f8be9fca88801e1b822d418271879.png)

---

<div class="post-metadata">

**Author:** ![niki](https://avatars.discourse-cdn.com/v4/letter/n/c57346/32.png) [@niki](https://discuss.elastic.co/u/niki)\
**Post date:** [February 12, 2020, 9:00am UTC](https://discuss.elastic.co/t/kafka-output-doesnt-use-winlogbeat-template/218310/2 "2020-02-12T09:00:20Z")

</div>

in case no one reply to this question. i kinda solved my problem by using redis instead of kafka.  
its working well in production so far. I needed elasticsearch in cluster mode and 5 redis and 5 logstash containers.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 11, 2020, 9:14am UTC](https://discuss.elastic.co/t/kafka-output-doesnt-use-winlogbeat-template/218310/3 "2020-03-11T09:14:27Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.