# Issue with filebeat kafka output lz4 compression

**URL:** https://discuss.elastic.co/t/issue-with-filebeat-kafka-output-lz4-compression/120427
**Category:** Beats
**Tags:** filebeat
**Created:** [February 19, 2018, 10:13am UTC](https://discuss.elastic.co/t/issue-with-filebeat-kafka-output-lz4-compression/120427 "2018-02-19T10:13:28Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![prabin\_meitei](https://avatars.discourse-cdn.com/v4/letter/p/c67d28/32.png) [@prabin\_meitei](https://discuss.elastic.co/u/prabin_meitei)
#### Post date: [February 19, 2018, 10:13am UTC](https://discuss.elastic.co/t/issue-with-filebeat-kafka-output-lz4-compression/120427/1 "2018-02-19T10:13:29Z")

</div>

I am trying to setup filebeat (6.2) with kafka output using lz4 compression, but always getting error as  
ERR Invalid kafka configuration: kafka: invalid configuration (lz4 compression requires Version \>= V0\_10\_0\_0)

My output configuration is:  
output.kafka:  
hosts: ["kafka1:10092","kafka2:10092","kafka3:10092"]  
topic: '%{[fields.log\_topic]}'  
partition.round\_robin:  
reachable\_only: false

required\_acks: 1  
compression: lz4  
max\_message\_bytes: 1000000  
version: 0.11.0.0

Any insight on what might be wrong would be very helpful

Thanks.

---

<div class="post-metadata">

### Author: ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [February 19, 2018, 3:41pm UTC](https://discuss.elastic.co/t/issue-with-filebeat-kafka-output-lz4-compression/120427/2 "2018-02-19T15:41:11Z")

</div>

Might be a bug. What happens if you set `version` to `0.10`?

---

<div class="post-metadata">

### Author: ![prabin\_meitei](https://avatars.discourse-cdn.com/v4/letter/p/c67d28/32.png) [@prabin\_meitei](https://discuss.elastic.co/u/prabin_meitei)
#### Post date: [February 19, 2018, 4:51pm UTC](https://discuss.elastic.co/t/issue-with-filebeat-kafka-output-lz4-compression/120427/3 "2018-02-19T16:51:24Z")

</div>

Got the same error. Another surprising thing, possibly another bug, is that if I specify the version as

> version: 0.10

I got the error as

> Exiting: error initializing publisher: unknown/unsupported kafka version '0.1' accessing 'output.kafka' (source:'filebeat.yml')

specifying version as (with quotes)

> version: "0.10"

gives the original error

> Invalid kafka configuration: kafka: invalid configuration (lz4 compression requires Version \>= V0\_10\_0\_0)

---

<div class="post-metadata">

### Author: ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [February 19, 2018, 5:53pm UTC](https://discuss.elastic.co/t/issue-with-filebeat-kafka-output-lz4-compression/120427/4 "2018-02-19T17:53:28Z")

</div>

This error is to be expected:

```auto
Exiting: error initializing publisher: unknown/unsupported kafka version '0.1' accessing 'output.kafka' (source:'filebeat.yml')

```

When using `1.10` without quotes, the YAML parser/spec will interpret the input as floating point value 🙂

With setting the version to `'0.10'`, you shouldn't get this error message. This is clearly a bug.

I see you've opened: [https://github.com/elastic/beats/issues/6414](https://github.com/elastic/beats/issues/6414)  
Thanks!

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 19, 2018, 5:54pm UTC](https://discuss.elastic.co/t/issue-with-filebeat-kafka-output-lz4-compression/120427/5 "2018-03-19T17:54:02Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
