# Set index depending on hostname/custom field

**URL:** https://discuss.elastic.co/t/set-index-depending-on-hostname-custom-field/110591
**Category:** Logstash
**Created:** [December 7, 2017, 1:42am UTC](https://discuss.elastic.co/t/set-index-depending-on-hostname-custom-field/110591 "2017-12-07T01:42:34Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![abd.wsu](https://avatars.discourse-cdn.com/v4/letter/a/b5a626/32.png) [@abd.wsu](https://discuss.elastic.co/u/abd.wsu)
#### Post date: [December 7, 2017, 1:42am UTC](https://discuss.elastic.co/t/set-index-depending-on-hostname-custom-field/110591/1 "2017-12-07T01:42:34Z")

</div>

Hello,

I am setting a custom field in filebeat for environment depending upon the server.  
I am then trying to use that field to create a seperate index depending on the field value. But am getting an error every time. Not sure what I am doing wrong. This is my LS config file

```
input {
  beats {
    port => 5044
}
}
output
{
if [env] == "DEV" {
    elasticsearch
    {
        hosts =>["10.204.16.105:9200"]
        index => "elk-dev"
        document_type => "log"
    }
    }
else if [env] == "QA" {
 elasticsearch
    {
        hosts =>["10.204.16.105:9200"]
        index => "elk-qa"
        document_type => "log"
    }
    }
else if [env] == "CERT" {
 elasticsearch
    {
        hosts =>["10.204.16.105:9200"]
        index => "elk-cert"
       document_type => "log"
    }
    }
}

```

I get some errors like below.

```
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.
Sending Logstash's logs to /etc/logstash-5.5.2/logs which is now configured via log4j2.properties
[2017-12-06T19:33:21,555][WARN][logstash.runner] --config.debug was specified, but log.level was not set to 'debug'! No config info will be logged.
[2017-12-06T19:33:22,060][INFO][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://10.204.16.105:9200/]}}
[2017-12-06T19:33:22,069][INFO][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://10.204.16.105:9200/, :path=>"/"}
[2017-12-06T19:33:22,141][WARN][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://10.204.16.105:9200/"}
[2017-12-06T19:33:22,144][INFO][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil}
[2017-12-06T19:33:22,179][INFO][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>50001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"_all"=>{"enabled"=>true, "norms"=>false}, "dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date", "include_in_all"=>false}, "@version"=>{"type"=>"keyword", "include_in_all"=>false}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
[2017-12-06T19:33:22,183][INFO][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//10.204.16.105:9200"]}
[2017-12-06T19:33:22,186][INFO][logstash.pipeline] Starting pipeline {"id"=>"main", "pipeline.workers"=>2, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>250}
[2017-12-06T19:33:22,565][INFO][logstash.inputs.beats] Beats inputs: Starting input listener {:address=>"0.0.0.0:5044"}
[2017-12-06T19:33:22,594][INFO][logstash.pipeline] Pipeline main started
[2017-12-06T19:33:22,678][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600}

```

Doesn't really look like there's an issue but i don't see any logs in kibana.  
Please help.

I can also provide the filebeat config.

```
filebeat:                                  
prospectors:                               
- input_type: log                          
  paths:                                   
    - /tmp/Messages/LOGQ1/*                
  encoding: plain                          
  fields_under_root: true                  
  exclude_lines: ["DMPMQMSG|^N|Queue|Qmgr"]
  document_type: qalog                     
  scan_frequency: 10s                      
  harvester_buffer_size: 16384             
  max_bytes: 10485760                      
  #index: iafelk-qa                        
  fields:                                  
    env: QA                                
  multiline.pattern: '<LogRecord'          
  multiline.negate: true                   
  multiline.match: after
```

---

<div class="post-metadata">

### Author: ![Albert\_zheng](https://avatars.discourse-cdn.com/v4/letter/a/3ec8ea/32.png) [@Albert\_zheng](https://discuss.elastic.co/u/Albert_zheng)
#### Post date: [December 12, 2017, 7:20am UTC](https://discuss.elastic.co/t/set-index-depending-on-hostname-custom-field/110591/2 "2017-12-12T07:20:55Z")

</div>

You can add custom field by mutate , it can be used in output.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/c/b/cb8d2a091801d5c448f3812baa036e41ca8539fa.png)

---

<div class="post-metadata">

### Author: ![tellus83](https://avatars.discourse-cdn.com/v4/letter/t/ecccb3/32.png) [@tellus83](https://discuss.elastic.co/u/tellus83)
#### Post date: [December 13, 2017, 9:16am UTC](https://discuss.elastic.co/t/set-index-depending-on-hostname-custom-field/110591/3 "2017-12-13T09:16:35Z")

</div>

If I'm not mistaking you need to add fields to your if line. "if [fields][env] == "DEV" {"

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 10, 2018, 9:17am UTC](https://discuss.elastic.co/t/set-index-depending-on-hostname-custom-field/110591/4 "2018-01-10T09:17:04Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
