# Filebeat - On-Demand Loading Compressed (.gz) Files from Stdin

**URL:** <https://discuss.elastic.co/t/filebeat-on-demand-loading-compressed-gz-files-from-stdin/184159>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [June 4, 2019, 11:22am UTC](https://discuss.elastic.co/t/filebeat-on-demand-loading-compressed-gz-files-from-stdin/184159 "2019-06-04T11:22:09Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![vasek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vasek/32/136636_2.png) [@vasek](https://discuss.elastic.co/u/vasek)\
**Post date:** [June 4, 2019, 11:22am UTC](https://discuss.elastic.co/t/filebeat-on-demand-loading-compressed-gz-files-from-stdin/184159/1 "2019-06-04T11:22:09Z")

</div>

If you want to load compressed files like .gz .zip etc. you can use **Filebeat** with input **stdin**.

```
vasek ~ $ filebeat version
filebeat version 7.0.1 (amd64), libbeat 7.0.1 [cbffb4dcc8d1d2b0ef2078cb7d7546092ee86e57 built 2019-04-29 12:09:21 +0000 UTC]

```

This command creates file /etc/filebeat/filebeat-stdin.yml.

```
cat << "EOF" > /etc/filebeat/filebeat-stdin.yml
filebeat.inputs:
- type: stdin
  fields_under_root: true
  fields:
    redis-key: "stdin-nazev-aplikace"
    application.name: "nazev-aplikace"
    application.environment: "production"
    service.type: "application"
    service.name: "tomcat"
logging.level: warning
logging.to_files: true
logging.to_syslog: false
logging.files:
  path: /var/log/filebeat
  name: filebeat
  keepfiles: 1
  permissions: 0640
  rotateeverybytes: 10485760
  metrics.enabled: false
processors:
 - drop_fields:
     fields: ["fields", "which", "you", "want", "to", "drop"]
queue.mem:
  events: 4096
  flush.timeout: 10
  flush.min.events: 2048
output.redis:
  hosts: ["localhost:6379"]
  key: "%{[redis-key]:unknown}"
EOF

```

- Filebeat **reads data from STDIN**
- Some **fields** will be **added** to document
- Some **fields** can be **removed** from document
- Data will be sent to **Redis** queue = **output**

There is command for loading compress files to filebeat stdin:

```
 zcat oracle-logs.2019-05-*.log.gz | filebeat -e -c /etc/filebeat/filebeat-stdin.yml

```

I hope it will be useful for someone.

---

<div class="post-metadata">

**Author:** ![Michael\_Madden](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/michael_madden/32/46640_2.png) [@Michael\_Madden](https://discuss.elastic.co/u/Michael_Madden)\
**Post date:** [June 27, 2019, 6:42pm UTC](https://discuss.elastic.co/t/filebeat-on-demand-loading-compressed-gz-files-from-stdin/184159/2 "2019-06-27T18:42:09Z")

</div>

Hello, thanks for reaching out and providing a solution for loading compressed files. I'm certain this will help someone in the future. Feel free to let us know if you have questions.

Thanks.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 25, 2019, 6:42pm UTC](https://discuss.elastic.co/t/filebeat-on-demand-loading-compressed-gz-files-from-stdin/184159/3 "2019-07-25T18:42:11Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.