# Shipping logs from multiple log files from single instance of filebeat to logstash

**URL:** <https://discuss.elastic.co/t/shipping-logs-from-multiple-log-files-from-single-instance-of-filebeat-to-logstash/192400>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [July 26, 2019, 10:10am UTC](https://discuss.elastic.co/t/shipping-logs-from-multiple-log-files-from-single-instance-of-filebeat-to-logstash/192400 "2019-07-26T10:10:02Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Eva](https://avatars.discourse-cdn.com/v4/letter/e/dbc845/32.png) [@Eva](https://discuss.elastic.co/u/Eva)\
**Post date:** [July 26, 2019, 10:10am UTC](https://discuss.elastic.co/t/shipping-logs-from-multiple-log-files-from-single-instance-of-filebeat-to-logstash/192400/1 "2019-07-26T10:10:02Z")

</div>

Hello

# I have a setup where i send only one log file like /abc/example.log to logstash instance like below:

filebeat.prospectors:

- paths:
  - /abc/example.log  
fields:  
app\_suite: cba  
application: abc

name: "xyz"

output.logstash:  
template.name: "filebeat"  
template.path: "filebeat.template.json"  
hosts: ["ip:5044"]

======================

In the same directory abc, i also have rolled logs from example.log compressed into the zip files to manage the storage. So the zipped files will contain old logs and fresh logs will be in example.log file.

I would like to know:  
If I can also ship these zipped files to logstash along with example.log file as I want old data also to be visible in elk and if it is possible, how do i do it. OR should I retain all the logs in example.log file with no rolling of logs to zipped folder?

Thanks

---

<div class="post-metadata">

**Author:** ![admlko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/admlko/32/21787_2.png) [@admlko](https://discuss.elastic.co/u/admlko)\
**Post date:** [July 26, 2019, 10:29am UTC](https://discuss.elastic.co/t/shipping-logs-from-multiple-log-files-from-single-instance-of-filebeat-to-logstash/192400/2 "2019-07-26T10:29:30Z")

</div>

I'd probably just concanate the old files into one and read it as one off.

---

<div class="post-metadata">

**Author:** ![Eva](https://avatars.discourse-cdn.com/v4/letter/e/dbc845/32.png) [@Eva](https://discuss.elastic.co/u/Eva)\
**Post date:** [July 26, 2019, 11:16am UTC](https://discuss.elastic.co/t/shipping-logs-from-multiple-log-files-from-single-instance-of-filebeat-to-logstash/192400/3 "2019-07-26T11:16:32Z")

</div>

Thanks for the response. You mean to say concatenating files is the only option here?

---

<div class="post-metadata">

**Author:** ![Eva](https://avatars.discourse-cdn.com/v4/letter/e/dbc845/32.png) [@Eva](https://discuss.elastic.co/u/Eva)\
**Post date:** [July 26, 2019, 12:50pm UTC](https://discuss.elastic.co/t/shipping-logs-from-multiple-log-files-from-single-instance-of-filebeat-to-logstash/192400/4 "2019-07-26T12:50:56Z")

</div>

Or i'm open to other options available here

---

<div class="post-metadata">

**Author:** ![admlko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/admlko/32/21787_2.png) [@admlko](https://discuss.elastic.co/u/admlko)\
**Post date:** [July 26, 2019, 12:55pm UTC](https://discuss.elastic.co/t/shipping-logs-from-multiple-log-files-from-single-instance-of-filebeat-to-logstash/192400/5 "2019-07-26T12:55:21Z")

</div>

Certainly not. I’m saying that would be the fastest way for me.

---

<div class="post-metadata">

**Author:** ![Eva](https://avatars.discourse-cdn.com/v4/letter/e/dbc845/32.png) [@Eva](https://discuss.elastic.co/u/Eva)\
**Post date:** [July 26, 2019, 3:36pm UTC](https://discuss.elastic.co/t/shipping-logs-from-multiple-log-files-from-single-instance-of-filebeat-to-logstash/192400/6 "2019-07-26T15:36:52Z")

</div>

Also just curious to know what would be the other options

---

<div class="post-metadata">

**Author:** ![admlko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/admlko/32/21787_2.png) [@admlko](https://discuss.elastic.co/u/admlko)\
**Post date:** [July 29, 2019, 6:17am UTC](https://discuss.elastic.co/t/shipping-logs-from-multiple-log-files-from-single-instance-of-filebeat-to-logstash/192400/7 "2019-07-29T06:17:15Z")

</div>

Why not create another directoy and extract all old logs there and then read that directory in?

---

<div class="post-metadata">

**Author:** ![Eva](https://avatars.discourse-cdn.com/v4/letter/e/dbc845/32.png) [@Eva](https://discuss.elastic.co/u/Eva)\
**Post date:** [August 2, 2019, 12:18pm UTC](https://discuss.elastic.co/t/shipping-logs-from-multiple-log-files-from-single-instance-of-filebeat-to-logstash/192400/8 "2019-08-02T12:18:12Z")

</div>

Sorry for the delayed response. We have implemented the same option.

Thanks!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 30, 2019, 12:18pm UTC](https://discuss.elastic.co/t/shipping-logs-from-multiple-log-files-from-single-instance-of-filebeat-to-logstash/192400/9 "2019-08-30T12:18:25Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
