# Using Filebeat Modules with Logstash and Differentiating Indices in Elasticsearch
**URL:** https://discuss.elastic.co/t/using-filebeat-modules-with-logstash-and-differentiating-indices-in-elasticsearch/351438
**Category:** Beats
**Tags:** beats-module
**Created:** [January 19, 2024, 11:45am UTC](https://discuss.elastic.co/t/using-filebeat-modules-with-logstash-and-differentiating-indices-in-elasticsearch/351438 "2024-01-19T11:45:11Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@yago82](https://discuss.elastic.co/u/yago82)
#### Post date: [January 19, 2024, 11:45am UTC](https://discuss.elastic.co/t/using-filebeat-modules-with-logstash-and-differentiating-indices-in-elasticsearch/351438/1 "2024-01-19T11:45:11Z")
Hi,
I am currently using the Apache module in Filebeat to process my Apache logs. My setup involves sending these logs to Logstash and then to Elasticsearch.
I understand from the documentation ([Working with Filebeat Modules | Logstash Reference [8.12] | Elastic](https://www.elastic.co/guide/en/logstash/current/filebeat-modules.html)) that it's possible to use Filebeat modules with Logstash, but some extra setup is required. Could someone please clarify what this extra setup involves?
Additionally, if I enable two or more modules in Filebeat, how can I differentiate the indices in Elasticsearch and associate the respective pipelines with them?
Any guidance would be greatly appreciated. Thank you.
---
### Author:  [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [January 19, 2024, 1:06pm UTC](https://discuss.elastic.co/t/using-filebeat-modules-with-logstash-and-differentiating-indices-in-elasticsearch/351438/2 "2024-01-19T13:06:35Z")
> [@yago82](#):
>
> Could someone please clarify what this extra setup involves?
Did you check the follow-up documentation on the same page? This [one](https://www.elastic.co/guide/en/logstash/current/use-ingest-pipelines.html), it explains how to configure it.
Basically you need to load the ingest pipelines in Elasticsearch and configure your logstash output to use them.
---
### Author:  [@yago82](https://discuss.elastic.co/u/yago82)
#### Post date: [January 19, 2024, 1:49pm UTC](https://discuss.elastic.co/t/using-filebeat-modules-with-logstash-and-differentiating-indices-in-elasticsearch/351438/3 "2024-01-19T13:49:30Z")
Hi Leandro,
Thank you for your response.
Unfortunately, I cannot directly access the servers where Filebeat is installed to reach Elasticsearch. For this reason, I need to go through Logstash.
So, I was wondering how I can load the ingest pipelines. Would it be possible, in your opinion, to do this through the installation of the relevant integration assets?
Also, considering this configuration:
```auto
index => "%{[@metadata][beat]}-%{[@metadata][version]}"
```
taken from the documentation, how would I create custom indices, for example, for Apache Access and Apache Error? From this configuration, it seems that the indices would be grouped, is that correct?
Regards
---
### Author:  [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [January 19, 2024, 2:01pm UTC](https://discuss.elastic.co/t/using-filebeat-modules-with-logstash-and-differentiating-indices-in-elasticsearch/351438/4 "2024-01-19T14:01:18Z")
> [@yago82](#):
>
> Unfortunately, I cannot directly access the servers where Filebeat is installed to reach Elasticsearch. For this reason, I need to go through Logstash.
>
> So, I was wondering how I can load the ingest pipelines. Would it be possible, in your opinion, to do this through the installation of the relevant integration assets?
You may run the filebeat setup process from any server, you just need a filebeat instance that is able to connect to Elasticsearch to load the ingest pipelnes.
For example you can install a filebeat on your Logstash server just to setup the ingest pipelines.
> [@yago82](#):
>
> From this configuration, it seems that the indices would be grouped, is that correct?
Yes, this configuration will store everything in a data stream named `filebeat-version`.
You can change the index name in logstash, but you will also probably need to change the `setup.template.name` and `setup.template.pattern` in the `filebeat.yml` of the filebeat instance you will use to load the intest pipelines.
Not sure how this will work as I do not use filebeat with custom indices, you will need to test it.
---
### Author:  [@yago82](https://discuss.elastic.co/u/yago82)
#### Post date: [January 19, 2024, 2:21pm UTC](https://discuss.elastic.co/t/using-filebeat-modules-with-logstash-and-differentiating-indices-in-elasticsearch/351438/5 "2024-01-19T14:21:26Z")
Hi,
Okay, perfect, excellent solution.
For the second part, I would need some advice. In your opinion, if I leverage `fileset.name` and `fileset.module`, could I create custom indices for each module and name?
Example Logstash configuration:
```auto
input {
beats {
port => 5044
}
output {
elasticsearch {
hosts => "https://061ab24010a2482e9d64729fdb0fd93a.us-east-1.aws.found.io:9243"
manage_template => false
index => "%{[fileset][module]}-%{[fileset][name]}"
action => "create"
user => "elastic"
password => "secret"
}
}
```
In this example, Logstash will dynamically create indices based on the combination of `fileset.module` and `fileset.name`
Regards
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 16, 2024, 4:21pm UTC](https://discuss.elastic.co/t/using-filebeat-modules-with-logstash-and-differentiating-indices-in-elasticsearch/351438/6 "2024-02-16T16:21:49Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.