# Multiple inputs and outputs in logstash conf file

**URL:** <https://discuss.elastic.co/t/multiple-inputs-and-outputs-in-logstash-conf-file/171025>\
**Category:** Logstash\
**Created:** [March 6, 2019, 5:12am UTC](https://discuss.elastic.co/t/multiple-inputs-and-outputs-in-logstash-conf-file/171025 "2019-03-06T05:12:04Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![sak6070](https://avatars.discourse-cdn.com/v4/letter/s/5e9695/32.png) [@sak6070](https://discuss.elastic.co/u/sak6070)\
**Post date:** [March 6, 2019, 5:12am UTC](https://discuss.elastic.co/t/multiple-inputs-and-outputs-in-logstash-conf-file/171025/1 "2019-03-06T05:12:04Z")

</div>

I have included multiple inputs and outputs in my logstash conf file (without filter for now).  
I have also created different indexes for each input. I am not able to see all the logs on kibana , also indices are not visible. can anybody suggest what could be the possible reason.

---

<div class="post-metadata">

**Author:** ![hermann102](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hermann102/32/41615_2.png) [@hermann102](https://discuss.elastic.co/u/hermann102)\
**Post date:** [March 6, 2019, 10:01am UTC](https://discuss.elastic.co/t/multiple-inputs-and-outputs-in-logstash-conf-file/171025/2 "2019-03-06T10:01:06Z")

</div>

This means that, no index has been created.  
There is no error when you start logstash?  
How does logstash get the data, Filebeat or other?

You can check if your indexes exist with this, without kibana.

> curl localhost:9200/\_cat/indices?v

If this command returns your indexes in the terminal, it is because they exist.

---

<div class="post-metadata">

**Author:** ![sak6070](https://avatars.discourse-cdn.com/v4/letter/s/5e9695/32.png) [@sak6070](https://discuss.elastic.co/u/sak6070)\
**Post date:** [March 7, 2019, 5:14am UTC](https://discuss.elastic.co/t/multiple-inputs-and-outputs-in-logstash-conf-file/171025/3 "2019-03-07T05:14:03Z")

</div>

logstash is getting data using filebeat. Below is my output file. mitrologs index is getting created but not the others.

output {  
if "scheduledwork" in [tags] {  
elasticsearch {  
hosts =\> ["[http://10.238.114.142:9200](http://10.238.114.142:9200)"]  
index =\> "scheduledwork-%{+YYYY-MM-dd}"  
}  
}  
if "emitroLog" in [tags] {  
elasticsearch {  
hosts =\> ["[http://10.238.114.142:9200](http://10.238.114.142:9200)"]  
index =\> "emitrolog-%{+YYYY-MM-dd}"  
document\_type =\> "\_doc"  
}  
}  
else {  
elasticsearch {  
hosts =\> "10.238.114.142:9200"  
index =\> "mitrologs-%{+YYYY.MM.dd}"  
}  
}  
stdout { codec =\> rubydebug }  
}

---

<div class="post-metadata">

**Author:** ![hermann102](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hermann102/32/41615_2.png) [@hermann102](https://discuss.elastic.co/u/hermann102)\
**Post date:** [March 7, 2019, 6:52am UTC](https://discuss.elastic.co/t/multiple-inputs-and-outputs-in-logstash-conf-file/171025/4 "2019-03-07T06:52:50Z")

</div>

Please format you code.

> output {
> 
> if "scheduledwork" in [tags] {
> 
> elasticsearch {  
> hosts =\> ["[http://10.238.114.142:9200](http://10.238.114.142:9200)"]  
> index =\> "scheduledwork-%{+YYYY-MM-dd}"  
> }  
> }
> 
> if "emitroLog" in [tags] {  
> elasticsearch {  
> hosts =\> ["[http://10.238.114.142:9200](http://10.238.114.142:9200)"]  
> index =\> "emitrolog-%{+YYYY-MM-dd}"  
> document\_type =\> "\_doc"  
> }
> 
> }else {
> 
> elasticsearch {  
> hosts =\> "10.238.114.142:9200"  
> index =\> "mitrologs-%{+YYYY.MM.dd}"  
> }
> 
> }  
> stdout { codec =\> rubydebug }  
> }

This may mean that your **emitroLog** and **scheduledwork** tags do not exist at logstash level or are named differently. Can you share your filebeat configuration file?

---

<div class="post-metadata">

**Author:** ![sak6070](https://avatars.discourse-cdn.com/v4/letter/s/5e9695/32.png) [@sak6070](https://discuss.elastic.co/u/sak6070)\
**Post date:** [March 7, 2019, 7:35am UTC](https://discuss.elastic.co/t/multiple-inputs-and-outputs-in-logstash-conf-file/171025/5 "2019-03-07T07:35:42Z")

</div>

here is my filebeat.yml

#=========================== Filebeat inputs =============================

## filebeat.inputs:

```
enabled: true
paths:
  - /opt/application/spring-boot-tomcat-server/logs/scheduledwork.log
type: log

```

- enabled: true  
paths:
  - /opt/jboss/7.2.0/mitro/logs/emitroLog.log  
type: log

#============================= Filebeat modules ===============================

filebeat.config.modules:

# Glob pattern for configuration loading

path: ${path.config}/modules.d/\*.yml

# Set to true to enable config reloading

reload.enabled: false

# Period on which files under path should be checked for changes

#reload.period: 10s

#==================== Elasticsearch template setting ==========================

setup.template.settings:  
index.number\_of\_shards: 3  
#index.codec: best\_compression  
#\_source.enabled: false

#============================== Dashboards =====================================

# These settings control loading the sample dashboards to the Kibana index. Loading

# the dashboards is disabled by default and can be enabled either by setting the

# options here, or by using the `-setup` CLI flag or the `setup` command.

setup.dashboards.enabled: true

# The URL from where to download the dashboards archive. By default this URL

# has a value which is computed based on the Beat name and version. For released

# versions, this URL points to the dashboard archive on the [artifacts.elastic.co](http://artifacts.elastic.co)

# website.

#setup.dashboards.url:

#============================== Kibana =====================================

# Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API.

# This requires a Kibana endpoint configuration.

setup.kibana:

host: "10.238.114.142:5601"

#================================ Outputs =====================================

# Configure what output to use when sending the data collected by the beat.

#-------------------------- Elasticsearch output ------------------------------  
#output.elasticsearch:

# Array of hosts to connect to.

hosts: ["10.238.114.142:9200"]

#----------------------------- Logstash output --------------------------------  
output.logstash:

# The Logstash hosts

hosts: ["10.238.114.142:5044"]

# Optional SSL. By default is off.

# List of root certificates for HTTPS server verifications

#ssl.certificate\_authorities: ["/etc/pki/root/ca.pem"]

# Certificate for SSL client authentication

#ssl.certificate: "/etc/pki/client/cert.pem"

# Client Certificate Key

#ssl.key: "/etc/pki/client/cert.key"

#================================ Procesors =====================================

# Configure processors to enhance or manipulate events generated by the beat.

processors:

- add\_host\_metadata: ~
- add\_cloud\_metadata: ~

#================================ Logging =====================================

# Sets log level. The default log level is info.

# Available log levels are: error, warning, info, debug

logging.level: debug

---

<div class="post-metadata">

**Author:** ![hermann102](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hermann102/32/41615_2.png) [@hermann102](https://discuss.elastic.co/u/hermann102)\
**Post date:** [March 7, 2019, 8:38am UTC](https://discuss.elastic.co/t/multiple-inputs-and-outputs-in-logstash-conf-file/171025/6 "2019-03-07T08:38:58Z")

</div>

I had to do something like that that worked. I hope that it will bring you a solution.

```
#=========================== Filebeat inputs =============================

filebeat.inputs:

- type: log
  # Change to true to enable this prospector configuration.
  enabled: true
  fields: {log_type: "scheduledwork"}
  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    - /opt/application/spring-boot-tomcat-server/logs/scheduledwork.log
    
- type: log
  enabled: true
  fields: {log_type: "emitroLog"}
  paths:
   - /opt/jboss/7.2.0/mitro/logs/emitroLog.log
   
   
#============================= Filebeat modules ===============================

filebeat.config.modules:

#Glob pattern for configuration loading
path: ${path.config}/modules.d/*.yml

#Set to true to enable config reloading
reload.enabled: false

#Period on which files under path should be checked for changes
#reload.period: 10s

#==================== Elasticsearch template setting ==========================

setup.template.settings:
index.number_of_shards: 3
#index.codec: best_compression
#_source.enabled: false

#============================== Dashboards =====================================

#These settings control loading the sample dashboards to the Kibana index. Loading
   # the dashboards is disabled by default and can be enabled either by setting the
#options here, or by using the -setup CLI flag or the setup command.
#setup.dashboards.enabled: true

   # The URL from where to download the dashboards archive. By default this URL
   # has a value which is computed based on the Beat name and version. For released
   # versions, this URL points to the dashboard archive on the artifacts.elastic.co
#website.
#setup.dashboards.url:

#============================== Kibana =====================================

#Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API.
   # This requires a Kibana endpoint configuration.

setup.kibana:

host: "10.238.114.142:5601"

#================================ Outputs =====================================

#Configure what output to use when sending the data collected by the beat.
#-------------------------- Elasticsearch output ------------------------------
#output.elasticsearch:

#Array of hosts to connect to.
#hosts: ["10.238.114.142:9200"]

#----------------------------- Logstash output --------------------------------
#The Logstash hosts
output.logstash:
hosts: ["10.238.114.142:5044"]

#Optional SSL. By default is off.
#List of root certificates for HTTPS server verifications
#ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]

Certificate for SSL client authentication
#ssl.certificate: "/etc/pki/client/cert.pem"

Client Certificate Key
#ssl.key: "/etc/pki/client/cert.key"

#================================ Procesors =====================================

#Configure processors to enhance or manipulate events generated by the beat.
processors:

add_host_metadata: ~
add_cloud_metadata: ~
#================================ Logging =====================================

#Sets log level. The default log level is info.
#Available log levels are: error, warning, info, debug
logging.level: debug

```

I update the **Filebeat inputs block** by adding **fields** to tag each log collection data point. you will be able to logstash, do this.

```
 output {

  if [fields][log_type] == "scheduledwork"{
   ## Do instruction 
   }
   
   if [fields][log_type] == "emitroLog"{
   ## Do instruction 
   }else{
    ## Do instruction 
   }
 }

```

I also commented **Elasticsearch output** , if you transfer the data via Logstash.

---

<div class="post-metadata">

**Author:** ![sak6070](https://avatars.discourse-cdn.com/v4/letter/s/5e9695/32.png) [@sak6070](https://discuss.elastic.co/u/sak6070)\
**Post date:** [March 7, 2019, 9:20am UTC](https://discuss.elastic.co/t/multiple-inputs-and-outputs-in-logstash-conf-file/171025/7 "2019-03-07T09:20:18Z")

</div>

hey It is working now. Thanks

---

<div class="post-metadata">

**Author:** ![sak6070](https://avatars.discourse-cdn.com/v4/letter/s/5e9695/32.png) [@sak6070](https://discuss.elastic.co/u/sak6070)\
**Post date:** [March 29, 2019, 5:44am UTC](https://discuss.elastic.co/t/multiple-inputs-and-outputs-in-logstash-conf-file/171025/8 "2019-03-29T05:44:09Z")

</div>

Hi,  
I have a query  
Can we run multiple logstash conf files? If yes, then how?  
I have to import logs for 2 different applications on elk server.

---

<div class="post-metadata">

**Author:** ![hermann102](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hermann102/32/41615_2.png) [@hermann102](https://discuss.elastic.co/u/hermann102)\
**Post date:** [March 29, 2019, 5:53am UTC](https://discuss.elastic.co/t/multiple-inputs-and-outputs-in-logstash-conf-file/171025/9 "2019-03-29T05:53:08Z")

</div>

Hello,  
Yes you can.  
Use Multi pipelines configuration to do that  
[https://www.elastic.co/guide/en/logstash/6.4/multiple-pipelines.html](https://www.elastic.co/guide/en/logstash/6.4/multiple-pipelines.html)

---

<div class="post-metadata">

**Author:** ![sak6070](https://avatars.discourse-cdn.com/v4/letter/s/5e9695/32.png) [@sak6070](https://discuss.elastic.co/u/sak6070)\
**Post date:** [March 29, 2019, 6:14am UTC](https://discuss.elastic.co/t/multiple-inputs-and-outputs-in-logstash-conf-file/171025/10 "2019-03-29T06:14:59Z")

</div>

I am not able to add error code and error text in visualization. There is a question mark in front of selected fields. what does that mean? Can you help on this.

![image](https://us1.discourse-cdn.com/elastic/original/3X/c/1/c1082e00572a5503dba7cc4ab674f485567b14bd.png)

---

<div class="post-metadata">

**Author:** ![hermann102](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hermann102/32/41615_2.png) [@hermann102](https://discuss.elastic.co/u/hermann102)\
**Post date:** [March 29, 2019, 7:23am UTC](https://discuss.elastic.co/t/multiple-inputs-and-outputs-in-logstash-conf-file/171025/11 "2019-03-29T07:23:45Z")

</div>

Get help message from the question mark and displays here.  
You can have it by pointing your mouse on question mark.

---

<div class="post-metadata">

**Author:** ![sak6070](https://avatars.discourse-cdn.com/v4/letter/s/5e9695/32.png) [@sak6070](https://discuss.elastic.co/u/sak6070)\
**Post date:** [March 29, 2019, 9:05am UTC](https://discuss.elastic.co/t/multiple-inputs-and-outputs-in-logstash-conf-file/171025/12 "2019-03-29T09:05:25Z")

</div>

Thanks but there is no such option of help message. Is there any other solution?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 26, 2019, 9:05am UTC](https://discuss.elastic.co/t/multiple-inputs-and-outputs-in-logstash-conf-file/171025/13 "2019-04-26T09:05:29Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.