# How to Create different index pattern for different filebeat server?

**URL:** <https://discuss.elastic.co/t/how-to-create-different-index-pattern-for-different-filebeat-server/170287>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [February 28, 2019, 7:53am UTC](https://discuss.elastic.co/t/how-to-create-different-index-pattern-for-different-filebeat-server/170287 "2019-02-28T07:53:05Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Aashish\_Chugh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aashish_chugh/32/40129_2.png) [@Aashish\_Chugh](https://discuss.elastic.co/u/Aashish_Chugh)\
**Post date:** [February 28, 2019, 7:53am UTC](https://discuss.elastic.co/t/how-to-create-different-index-pattern-for-different-filebeat-server/170287/1 "2019-02-28T07:53:06Z")

</div>

Hi,

I want to create a new index pattern for multiple filebeat server , like i have two server one is app and second is db server, for now both the logs is storing in same index.

I try many ways to separate them but nothing is working , anyone here who can help me in this situation.

Below is my configuration file.

Logstash- conf

output {  
elasticsearch {  
hosts =\> localhost  
manage\_template =\> false  
index =\> "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"  
}  
stdout { codec =\> rubydebug }  
}

Filebeat-conf

###################### Filebeat Configuration Example #########################

# This file is an example configuration file highlighting only the most common

# options. The filebeat.reference.yml file from the same directory contains all the

# supported options with more comments. You can use it as a reference.

# 

# You can find the full configuration reference here:

# [https://www.elastic.co/guide/en/beats/filebeat/index.html](https://www.elastic.co/guide/en/beats/filebeat/index.html)

# For more available modules and options, please see the filebeat.reference.yml sample

# configuration file.

#=========================== Filebeat inputs =============================

filebeat.prospectors:

# Each - is an input. Most options can be set at the input level, so

# you can use different inputs for various configurations.

# Below are the input specific configurations.

- type: log  
multiline:  
multiline.pattern: '^# |;+$'  
multiline.negate: true  
multiline.match: after  
#multiline.max\_lines: 5  
#Change to true to enable this input configuration.  
enabled: true

# service: db\_log

fields\_under\_root: true

# Exclude lines. A list of regular expressions to match. It drops the lines that are

# matching any regular expression from the list.

#exclude\_lines: ['^DBG']

# Include lines. A list of regular expressions to match. It exports the lines that are

# matching any regular expression from the list.

#include\_lines: ['^ERR', '^WARN']

# Exclude files. A list of regular expressions to match. Filebeat drops the files that

# are matching any regular expression from the list. By default, no files are dropped.

#exclude\_files: ['.gz$']

# Optional additional fields. These fields can be freely picked

# to add additional information to the crawled log files for filtering

#fields:

# level: debug

# review: 1

### Multiline options

# Multiline can be used for log messages spanning multiple lines. This is common

# for Java Stack Traces or C-Line Continuation

# The regexp Pattern that has to be matched. The example pattern matches all lines starting with [

#multiline.pattern: ^[

# Defines if the pattern set under pattern should be negated or not. Default is false.

#multiline.negate: false

# Match can be set to "after" or "before". It is used to define if lines should be append to a pattern

# that was (not) matched before or after or as long as a pattern is not matched based on negate.

# Note: After is the equivalent to previous and before is the equivalent to to next in Logstash

#multiline.match: after

#============================= Filebeat modules ===============================

filebeat.config.modules:

# Glob pattern for configuration loading

path: ${path.config}/modules.d/\*.yml

# Set to true to enable config reloading

reload.enabled: false

# Period on which files under path should be checked for changes

#reload.period: 10s

#==================== Elasticsearch template setting ==========================

setup.template.settings:  
index.number\_of\_shards: 3  
#index.codec: best\_compression  
#\_source.enabled: false

#================================ General =====================================

# The name of the shipper that publishes the network data. It can be used to group

# all the transactions sent by a single shipper in the web interface.

#name:

# The tags of the shipper are included in their own field with each

# transaction published.

#tags: ["service-X", "web-tier"]

# Optional fields that you can specify to add additional information to the

# output.

#fields:  
env: prod

#============================== Dashboards =====================================

# These settings control loading the sample dashboards to the Kibana index. Loading

# the dashboards is disabled by default and can be enabled either by setting the

# options here, or by using the `-setup` CLI flag or the `setup` command.

#setup.dashboards.enabled: false

# The URL from where to download the dashboards archive. By default this URL

# has a value which is computed based on the Beat name and version. For released

# versions, this URL points to the dashboard archive on the [artifacts.elastic.co](http://artifacts.elastic.co)

# website.

#setup.dashboards.url:

#============================== Kibana =====================================

# Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API.

# This requires a Kibana endpoint configuration.

setup.kibana:

# Kibana Host

# Scheme and port can be left out and will be set to the default (http and 5601)

# In case you specify and additional path, the scheme is required: [http://localhost:5601/path](http://localhost:5601/path)

# IPv6 addresses should always be defined as: https://[2001:db8::1]:5601

#host: "localhost:5601"

# Kibana Space ID

# ID of the Kibana Space into which the dashboards should be loaded. By default,

# the Default Space will be used.

#space.id:

#============================= Elastic Cloud ==================================

# These settings simplify using filebeat with the Elastic Cloud ([https://cloud.elastic.co/](https://cloud.elastic.co/)).

# The cloud.id setting overwrites the `output.elasticsearch.hosts` and

# `setup.kibana.host` options.

# You can find the `cloud.id` in the Elastic Cloud web UI.

#cloud.id:

# The cloud.auth setting overwrites the `output.elasticsearch.username` and

# `output.elasticsearch.password` settings. The format is `<user>:<pass>`.

#cloud.auth:

#================================ Outputs =====================================

# Configure what output to use when sending the data collected by the beat.

#-------------------------- Elasticsearch output ------------------------------  
#output.elasticsearch:

# Array of hosts to connect to.

# hosts: ["localhost:9200"]

# Optional protocol and basic auth credentials.

#protocol: "https"  
#username: "elastic"  
#password: "changeme"

#----------------------------- Logstash output --------------------------------  
output.logstash:  
hosts: ["x.x.x.x:5044"]

# Optional SSL. By default is off.

# List of root certificates for HTTPS server verifications

#ssl.certificate\_authorities: ["/etc/pki/root/ca.pem"]

# Configure processors to enhance or manipulate events generated by the beat.

processors:

- add\_host\_metadata: ~
- add\_cloud\_metadata: ~

---

<div class="post-metadata">

**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\
**Post date:** [February 28, 2019, 7:58am UTC](https://discuss.elastic.co/t/how-to-create-different-index-pattern-for-different-filebeat-server/170287/2 "2019-02-28T07:58:13Z")

</div>

Could you please format your configurations using `</>`?

---

<div class="post-metadata">

**Author:** ![harshbajaj16](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshbajaj16/32/44970_2.png) [@harshbajaj16](https://discuss.elastic.co/u/harshbajaj16)\
**Post date:** [February 28, 2019, 9:23am UTC](https://discuss.elastic.co/t/how-to-create-different-index-pattern-for-different-filebeat-server/170287/3 "2019-02-28T09:23:51Z")

</div>

Hi @Aashish_Chugh,

You can add the tags in your filebeat configuration for each node. Below is the configuration for the same.

```
# The tags of the shipper are included in their own field with each

# transaction published.

tags: ["apps"]

```

And your logs come with the tags field and you can add the same in logstash o/p like below example. you can modify the same as per your requirement.

> ```
> output {
> if "apps" in [tags] {
> elasticsearch {
> hosts => ["ES1"]
> index => "apps-%{+YYYY.MM.dd}"
> }
> }
> else if "DB" in [tags] {
> elasticsearch {
> hosts => ["ES1"]
> index => "DB-%{+YYYY.MM.dd}"
> }
> }
> else {
> elasticsearch {
> hosts => ["ES1"]
> index => "other-%{+YYYY.MM.dd}"
> }
> }
> }
> 
> ```

Please do let me know case of you have any further query.

Regards,  
Harsh Bajaj

---

<div class="post-metadata">

**Author:** ![Aashish\_Chugh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aashish_chugh/32/40129_2.png) [@Aashish\_Chugh](https://discuss.elastic.co/u/Aashish_Chugh)\
**Post date:** [February 28, 2019, 11:43am UTC](https://discuss.elastic.co/t/how-to-create-different-index-pattern-for-different-filebeat-server/170287/4 "2019-02-28T11:43:08Z")

</div>

Thanks bro this is working, i was looking for this feature since last few days.

can i apply same in my filter section like.

filter {  
if "apps" in [tags] {  
grok {  
match =\> ["request", "%{URIPROTO:http\_protocol}"] } } }

Please let me know.

---

<div class="post-metadata">

**Author:** ![harshbajaj16](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshbajaj16/32/44970_2.png) [@harshbajaj16](https://discuss.elastic.co/u/harshbajaj16)\
**Post date:** [February 28, 2019, 12:29pm UTC](https://discuss.elastic.co/t/how-to-create-different-index-pattern-for-different-filebeat-server/170287/5 "2019-02-28T12:29:58Z")

</div>

Yes, You can use the same in filter section also.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 28, 2019, 12:29pm UTC](https://discuss.elastic.co/t/how-to-create-different-index-pattern-for-different-filebeat-server/170287/6 "2019-03-28T12:29:59Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.