# Run multiple instance of logstash

**URL:** https://discuss.elastic.co/t/run-multiple-instance-of-logstash/92456
**Category:** Logstash
**Created:** [July 10, 2017, 11:35am UTC](https://discuss.elastic.co/t/run-multiple-instance-of-logstash/92456 "2017-07-10T11:35:28Z")
**Posts on this page:** 15
**Page:** 1

<div class="post-metadata">

### Author: ![karim](https://avatars.discourse-cdn.com/v4/letter/k/46a35a/32.png) [@karim](https://discuss.elastic.co/u/karim)
#### Post date: [July 10, 2017, 11:35am UTC](https://discuss.elastic.co/t/run-multiple-instance-of-logstash/92456/1 "2017-07-10T11:35:28Z")

</div>

i have this Error : FATAL logstash.runner - Logstash could not be started because there is already another instance using the configured data directory. If you wish to run multiple instances, you must change the "path.data" setting.  
help please

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [July 10, 2017, 11:37am UTC](https://discuss.elastic.co/t/run-multiple-instance-of-logstash/92456/2 "2017-07-10T11:37:07Z")

</div>

What is not clear around the error message?

---

<div class="post-metadata">

### Author: ![karim](https://avatars.discourse-cdn.com/v4/letter/k/46a35a/32.png) [@karim](https://discuss.elastic.co/u/karim)
#### Post date: [July 10, 2017, 11:40am UTC](https://discuss.elastic.co/t/run-multiple-instance-of-logstash/92456/3 "2017-07-10T11:40:57Z")

</div>

I can not solve this error

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [July 10, 2017, 11:58am UTC](https://discuss.elastic.co/t/run-multiple-instance-of-logstash/92456/4 "2017-07-10T11:58:18Z")

</div>

Either you run one instance of Logstash with all configs, or you need to install separate instances with different data directories.

---

<div class="post-metadata">

### Author: ![karim](https://avatars.discourse-cdn.com/v4/letter/k/46a35a/32.png) [@karim](https://discuss.elastic.co/u/karim)
#### Post date: [July 10, 2017, 12:14pm UTC](https://discuss.elastic.co/t/run-multiple-instance-of-logstash/92456/5 "2017-07-10T12:14:38Z")

</div>

And how can i to do it ?

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [July 10, 2017, 12:16pm UTC](https://discuss.elastic.co/t/run-multiple-instance-of-logstash/92456/6 "2017-07-10T12:16:44Z")

</div>

What is it you are trying to do? Which operating system are you using? How did you install Logstash?

---

<div class="post-metadata">

### Author: ![karim](https://avatars.discourse-cdn.com/v4/letter/k/46a35a/32.png) [@karim](https://discuss.elastic.co/u/karim)
#### Post date: [July 10, 2017, 1:12pm UTC](https://discuss.elastic.co/t/run-multiple-instance-of-logstash/92456/7 "2017-07-10T13:12:11Z")

</div>

I use centos and j installed logstash from RPM source I have 3 database source SQL server, postgres and oracle database and I have 2 csv file I must index them in elasticsearch

---

<div class="post-metadata">

### Author: ![inhinyera16](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/inhinyera16/32/61625_2.png) [@inhinyera16](https://discuss.elastic.co/u/inhinyera16)
#### Post date: [July 11, 2017, 2:36am UTC](https://discuss.elastic.co/t/run-multiple-instance-of-logstash/92456/8 "2017-07-11T02:36:43Z")

</div>

This is how i start my logstash specifying path.data

{logstash\_home}/bin/logstash -f /usr/share/logstash/kafka-solr-pipeline.conf --config.reload.automatic --path.data /usr/share/logstash/data/1

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [July 11, 2017, 5:30am UTC](https://discuss.elastic.co/t/run-multiple-instance-of-logstash/92456/9 "2017-07-11T05:30:47Z")

</div>

If you install using RPM I do not think there is any automatic way to create multiple services, so you may be better off separating the processing flows in your configuration files using conditionals and have a single Logstash instance process them all.

---

<div class="post-metadata">

### Author: ![karim](https://avatars.discourse-cdn.com/v4/letter/k/46a35a/32.png) [@karim](https://discuss.elastic.co/u/karim)
#### Post date: [July 11, 2017, 9:49am UTC](https://discuss.elastic.co/t/run-multiple-instance-of-logstash/92456/10 "2017-07-11T09:49:42Z")

</div>

how can i do this

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [July 11, 2017, 9:56am UTC](https://discuss.elastic.co/t/run-multiple-instance-of-logstash/92456/11 "2017-07-11T09:56:57Z")

</div>

Update all your config files with tags and associated [conditionals](https://www.elastic.co/guide/en/logstash/5.5/event-dependent-configuration.html#conditionals) so that they can be merged into a single config without issues, and then put them all into the Logstash [config directory](https://www.elastic.co/guide/en/logstash/5.5/dir-layout.html), which should be `/etc/logstash/conf.d` by default.

---

<div class="post-metadata">

### Author: ![karim](https://avatars.discourse-cdn.com/v4/letter/k/46a35a/32.png) [@karim](https://discuss.elastic.co/u/karim)
#### Post date: [July 11, 2017, 10:13am UTC](https://discuss.elastic.co/t/run-multiple-instance-of-logstash/92456/12 "2017-07-11T10:13:17Z")

</div>

you can help me do it ?

> input {  
> file {  
> path =\> "/etc/logstash/conf.d/Produit.csv"  
> start\_position =\> "beginning"  
> sincedb\_path =\> "/dev/null"  
> }  
> }  
> filter {  
> csv {  
> separator =\> ";"  
> columns =\>["ProductId","Name","Description"]  
> }  
> }  
> output {  
> elasticsearch {  
> hosts =\> "192.168.10.150"  
> workers =\> 1  
> index =\> "Produit"  
> document\_type =\> "Produit"  
> }

> stdout {codec =\> rubydebug}  
> input {  
> file {  
> path =\> "/etc/logstash/conf.d/Contrat.csv"  
> start\_position =\> "beginning"  
> sincedb\_path =\> "/dev/null"  
> }  
> }  
> filter {  
> csv {  
> separator =\> ";"  
> columns =\>["......."]  
> }  
> }  
> output {  
> elasticsearch {  
> hosts =\> "192.168.10.150"  
> workers =\> 1  
> index =\> "Contrat"  
> document\_type =\> "Contrat"  
> }  
> stdout {codec =\> rubydebug}

```
jdbc {

        jdbc_connection_string => "jdbc:postgresql://192.168.10.100:5432/postgres"

        jdbc_user => "postgres"
        jdbc_validate_connection => true
        jdbc_driver_library => "/etc/logstash/postgresql-42.0.0.jar"

        jdbc_driver_class => "org.postgresql.Driver"

        statement => "SELECT * from clientb2b"
}

}
output {
    elasticsearch {

        index => "clientbb"
        document_type => "clientbb"
        document_id => "%{telephone}"
        hosts => "192.168.10.150"

```

> input {
> 
> jdbc {
> 
> jdbc\_driver\_library =\> "/opt/ojdbc6.jar"
> 
> jdbc\_driver\_class =\> "Java::oracle.jdbc.driver.OracleDriver"
> 
> jdbc\_connection\_string =\> "jdbc:oracle:thin:@192.168.10.137:1521/sv00"
> 
> jdbc\_user =\> "COSWIN"
> 
> jdbc\_password =\> "COSWIN"
> 
> #jdbc\_validate\_connection =\> TRUE
> 
> statement =\> "SELECT \* from contact\_client"
> 
> }
> 
> }
> 
> output {
> 
> elasticsearch {
> 
> action =\> "index"
> 
> hosts =\> "192.168.10.150:9200"
> 
> index =\> "client\_contact"
> 
> document\_type =\> "client\_contact"
> 
> }  
> stdout { codec =\> rubydebug }  
> }

> ```
> input {
> jdbc {
> jdbc_driver_library => "/etc/logstash/sqljdbc42.jar"
> jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver"
> jdbc_connection_string => "jdbc:sqlserver://172.17.1.100;user=sa;password=root;"
> 
> ```

> ```
> jdbc_user => "sa"
> jdbc_password => "root"
> statement_filepath => "/etc/logstash/conf.d/query.sql"
>     
> }
> }
> filter {
> }
> output {
> elasticsearch {
>  	
> #protocol => "http"
> index => "incident"
> document_type => "incident"
> document_id => "%{incidentid}"
> hosts => "192.168.10.150"
> }
> stdout { codec => rubydebug }
> 
> ```

> ```
> }
> 
> ```

---

<div class="post-metadata">

### Author: ![karim](https://avatars.discourse-cdn.com/v4/letter/k/46a35a/32.png) [@karim](https://discuss.elastic.co/u/karim)
#### Post date: [July 13, 2017, 9:10am UTC](https://discuss.elastic.co/t/run-multiple-instance-of-logstash/92456/13 "2017-07-13T09:10:51Z")

</div>

any help please

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [July 13, 2017, 9:14am UTC](https://discuss.elastic.co/t/run-multiple-instance-of-logstash/92456/14 "2017-07-13T09:14:42Z")

</div>

Easiest way is to add a distinct tag for each input using the `tags` parameter and then use conditionals to ensure that the correct filters and outputs are applied. Something like this:

```auto
input {
  jdbc {

    ...
    
    tags =>["jdbc1"]
  }
}

filter {
  if "jdbc1" in [tags] {

    ...
  
  }
}

output {
  if "jdbc1" in [tags] {
    elasticsearch {
  
      ...
  
    }
  }
}

```

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 10, 2017, 9:15am UTC](https://discuss.elastic.co/t/run-multiple-instance-of-logstash/92456/15 "2017-08-10T09:15:02Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
