# Filebeat: How to create multiple instances/pipelines on same VM

**URL:** <https://discuss.elastic.co/t/filebeat-how-to-create-multiple-instances-pipelines-on-same-vm/352999>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [February 10, 2024, 9:49pm UTC](https://discuss.elastic.co/t/filebeat-how-to-create-multiple-instances-pipelines-on-same-vm/352999 "2024-02-10T21:49:24Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![albus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/albus/32/131636_2.png) [@albus](https://discuss.elastic.co/u/albus)\
**Post date:** [February 10, 2024, 9:49pm UTC](https://discuss.elastic.co/t/filebeat-how-to-create-multiple-instances-pipelines-on-same-vm/352999/1 "2024-02-10T21:49:24Z")

</div>

Hello Elastic Community,

I hope everybody is doing great.

I am facing a problem and I cannot find any documentation or blog related to it. Problem statement is: **Is there a possibility to run multiple instances of Filebeat as a different process from different config files ? Each config file will read data from separate path and ship it to separate output. (Each instance should be completely independent of each other.)**

One solution I can think of is to use

```auto
/usr/bin/filebeat -c /path/to/each _filebeat_config

```

and define a service for each process.

But I am looking for something a little more "official".

Can anyone help me out by providing a different perspective or a documentation / blog I might have missed.

Thanks.

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [February 10, 2024, 11:49pm UTC](https://discuss.elastic.co/t/filebeat-how-to-create-multiple-instances-pipelines-on-same-vm/352999/2 "2024-02-10T23:49:59Z")

</div>

> [@albus](#):
>
> But I am looking for something a little more "official".

It is possible to run multiple instances of filebeat on the same server, but I don't think there is any official documentation on how to do it because it is more an infrastructure issue than a Filebeat issue.

What you find in the [documentation](https://www.elastic.co/guide/en/beats/filebeat/8.12/configuration-path.html#_data) is a note telling that when running multiple instances in the same server, they need to use a different data path, so it is not just pointing to a different config file, but you need to configure it to use a different data path.

So you would need to have a different `filebeat.yml` file for each instance using a different `path.data`.

---

<div class="post-metadata">

**Author:** ![albus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/albus/32/131636_2.png) [@albus](https://discuss.elastic.co/u/albus)\
**Post date:** [February 12, 2024, 3:30am UTC](https://discuss.elastic.co/t/filebeat-how-to-create-multiple-instances-pipelines-on-same-vm/352999/3 "2024-02-12T03:30:35Z")

</div>

noted.  
is it possible to parse multiple configuration files through -c option in filebeat ?  
like:

filebeat -c conf1.yml -c conf2.yml ...... and so on.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [February 12, 2024, 3:43am UTC](https://discuss.elastic.co/t/filebeat-how-to-create-multiple-instances-pipelines-on-same-vm/352999/4 "2024-02-12T03:43:14Z")

</div>

> [@albus](#):
>
> filebeat -c conf1.yml -c conf2.yml ...... and so on.

No 🙂

If you want to run more than 1 filebeat create separate directories and the use the tar.gz distribution in each that is the easiest/ safest way.

---

<div class="post-metadata">

**Author:** ![albus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/albus/32/131636_2.png) [@albus](https://discuss.elastic.co/u/albus)\
**Post date:** [February 12, 2024, 7:20pm UTC](https://discuss.elastic.co/t/filebeat-how-to-create-multiple-instances-pipelines-on-same-vm/352999/5 "2024-02-12T19:20:18Z")

</div>

Thanks.  
I am guessing I will have to provide different data and home paths for different instances for complete isolation.  
Is that correct ?  
Or the default paths (except config) will do the trick ?

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [February 12, 2024, 7:51pm UTC](https://discuss.elastic.co/t/filebeat-how-to-create-multiple-instances-pipelines-on-same-vm/352999/6 "2024-02-12T19:51:38Z")

</div>

> [@albus](#):
>
> I am guessing I will have to provide different data and home paths for different instances for complete isolation.  
> Is that correct ?  
> Or the default paths (except config) will do the trick ?

Little unclear what you are asking, so to be Precise if you put tar.gz distributions in their own directories and start from the command `./filebeat` the data paths will be relative to each directory so defaults will/should work... I believe

Quick test should show you...

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 11, 2024, 9:52pm UTC](https://discuss.elastic.co/t/filebeat-how-to-create-multiple-instances-pipelines-on-same-vm/352999/7 "2024-03-11T21:52:11Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
