# Filebeat compress output to file not working **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [January 7, 2018, 10:31pm UTC](https://discuss.elastic.co/t/filebeat-compress-output-to-file-not-working/114434 "2018-01-07T22:31:48Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Omair\_Khalid](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/omair_khalid/32/23533_2.png) [@Omair\_Khalid](https://discuss.elastic.co/u/Omair_Khalid)\ **Post date:** [January 7, 2018, 10:31pm UTC](https://discuss.elastic.co/t/filebeat-compress-output-to-file-not-working/114434/1 "2018-01-07T22:31:48Z")
Hello, I am reading logs from a log file (as below) and then emitting the output into another file. Note: i am not emitting the output to ES or kafka e.t.c. But another file on some other location. I have been able to achieve the normal flow, but i am unable to perform any compression. As per documentation, we can set the compression\_level field, but its not working. Can some one help me out? My Filebeat config file: #=========================== Filebeat prospectors ============================= filebeat.prospectors: - type: log enabled: true paths: - C:/path/logfile.out and then i am emitting the output in another file as this. #================================ Outputs ===================================== # Configure what output to use when sending the data collected by the beat. output.file: path: "C:/someRemoteLocation/" compression\_level: 10 gzip: true filename: filebeat As you can see i have specified various tags like compression\_level and gzip e.t.c, but none of them seems to be working. ---
**Author:** ![adrisr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adrisr/32/25423_2.png) [@adrisr](https://discuss.elastic.co/u/adrisr)\ **Post date:** [January 8, 2018, 5:10am UTC](https://discuss.elastic.co/t/filebeat-compress-output-to-file-not-working/114434/2 "2018-01-08T05:10:24Z")
Hi Omair, Unfortunately, the `file` output doesn't support compression. This is only available at the `elasticsearch` and `logstash` outputs. Although the file output is currently intended for diagnostics, it might make sense to add support for this. If you're insterested please open a GitHub issue with the feature request [here](https://github.com/elastic/beats/issues), so we can discuss it further. ---
**Author:** ![Omair\_Khalid](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/omair_khalid/32/23533_2.png) [@Omair\_Khalid](https://discuss.elastic.co/u/Omair_Khalid)\ **Post date:** [January 8, 2018, 9:38am UTC](https://discuss.elastic.co/t/filebeat-compress-output-to-file-not-working/114434/3 "2018-01-08T09:38:08Z")
Hi, Thanks for your reply. Couple of quick question here. 1 - If we compress the output and send it to logstash, then who would decompress it. Because elasticsearch is just a storage place. Its not intelligent enough to decompress it. Or do we need another application to fetch the data from elastic again and decompress it. 2 - Any idea that in case of we do compression at filebeat, then how much extra CPU is consumed from normal processing. What is the effect on CPU if lets say compression level is 9.? ---
**Author:** ![adrisr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adrisr/32/25423_2.png) [@adrisr](https://discuss.elastic.co/u/adrisr)\ **Post date:** [January 8, 2018, 10:41am UTC](https://discuss.elastic.co/t/filebeat-compress-output-to-file-not-working/114434/4 "2018-01-08T10:41:55Z")
Compression is used only to send requests over the network. The events themselves are stored decompressed, as if no compression was used. About the CPU impact, you can have a look at this benchmarks, taking into account only the values for gzip, which is the library used in outputs: > **[Gzip vs Bzip2 vs XZ Performance Comparison](https://www.rootusers.com/gzip-vs-bzip2-vs-xz-performance-comparison/)** > > Gzip, Bzip2 and XZ are popular compression tools, but which performs best? Here we benchmark them and compare the trade off between compression ratio and speed. [https://tukaani.org/lzma/benchmarks.html](https://tukaani.org/lzma/benchmarks.html) However, these benchmarks are performed with large files. In the case of beats, where the events are usually small, I think a compression level of 1 is enough and will have little impact on the CPU. ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [January 8, 2018, 10:46pm UTC](https://discuss.elastic.co/t/filebeat-compress-output-to-file-not-working/114434/5 "2018-01-08T22:46:22Z")
Beats sends batches of events to Logstash and Elasticsearch. Compression happens on application network layer only, but the events themselves are not compressed. Beats uses gzip compression, which operates at 32KB blocks I think. Events are JSON encoded. Adding compression really reduces bytes being send over the network (at the cost of higher CPU usage). The default compression level of 3 is a quite good default I think (higher values don't add much more compression benefits, but increase CPU usage). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 5, 2018, 10:46pm UTC](https://discuss.elastic.co/t/filebeat-compress-output-to-file-not-working/114434/6 "2018-02-05T22:46:24Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.