# Use logstash as a central logging server for log files exported from various devices

**URL:** https://discuss.elastic.co/t/use-logstash-as-a-central-logging-server-for-log-files-exported-from-various-devices/330689
**Category:** Logstash
**Created:** [April 24, 2023, 9:34pm UTC](https://discuss.elastic.co/t/use-logstash-as-a-central-logging-server-for-log-files-exported-from-various-devices/330689 "2023-04-24T21:34:05Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![Arinjay\_Jain](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arinjay_jain/32/97753_2.png) [@Arinjay\_Jain](https://discuss.elastic.co/u/Arinjay_Jain)
#### Post date: [April 24, 2023, 9:34pm UTC](https://discuss.elastic.co/t/use-logstash-as-a-central-logging-server-for-log-files-exported-from-various-devices/330689/1 "2023-04-24T21:34:05Z")

</div>

Hi All,  
I want to use logstash as a central logging server for storing log files exported from various devices. The log files can contain structured as well as un-structured data. Also I would like to store binary files as well which can be gzipped before sending to logstash. Can someone suggest what should be the pipeline configuration to achieve this?

Thanks,  
Arinjay

---

<div class="post-metadata">

### Author: ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [April 24, 2023, 10:00pm UTC](https://discuss.elastic.co/t/use-logstash-as-a-central-logging-server-for-log-files-exported-from-various-devices/330689/2 "2023-04-24T22:00:24Z")

</div>

Hello,

> [@Arinjay\_Jain](#):
>
> I want to use logstash as a central logging server for storing log files exported from various devices.

Logstash does not store anything, it is an ETL tool that process data, it has many inputs, filters and outputs that enables you to receive/read logs from different sources, transform them and send to different outputs where they will be stored.

Probably you want to store your logs in Elasticsearch to be able to query your data.

> [@Arinjay\_Jain](#):
>
> The log files can contain structured as well as un-structured data.

This is no problem for logstash, but you may need to parse your logs to make it easier to search your data.

> [@Arinjay\_Jain](#):
>
> Also I would like to store binary files as well which can be gzipped before sending to logstash.

It does not make much sense to store binary data in elasticsearch as it is an search engine, it will just wast disk space.

> [@Arinjay\_Jain](#):
>
> Can someone suggest what should be the pipeline configuration to achieve this?

Your question is too broad, you need to provide examples of what devices you want to monitor, what your data looks like, there are plenty of posts in the forum as well with many examples.

---

<div class="post-metadata">

### Author: ![Arinjay\_Jain](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arinjay_jain/32/97753_2.png) [@Arinjay\_Jain](https://discuss.elastic.co/u/Arinjay_Jain)
#### Post date: [April 25, 2023, 6:21pm UTC](https://discuss.elastic.co/t/use-logstash-as-a-central-logging-server-for-log-files-exported-from-various-devices/330689/3 "2023-04-25T18:21:55Z")

</div>

Thanks @leandrojmp for the response. The requirement to store binary data is to be able to fetch process core files when a process crashes on the device. This obviously is not searchable in Elastic but is needed to analyse the reason behind the crash. The associated process logs stored in Elastic will help in this.

Another example of binary data is to periodically pull CPU, memory and disk I/O metrics from the device which an application like Linux perf can then use it to generate Flame graphs.

I have come across metric beats module which can ship this type of device metrics to logstash, but am not sure how to configure logstash to handle it.

Thanks,  
Arinjay

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 23, 2023, 6:22pm UTC](https://discuss.elastic.co/t/use-logstash-as-a-central-logging-server-for-log-files-exported-from-various-devices/330689/4 "2023-05-23T18:22:54Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
