# Create multiple indexs with multiple input in logstash

**URL:** <https://discuss.elastic.co/t/create-multiple-indexs-with-multiple-input-in-logstash/201779>\
**Category:** Logstash\
**Created:** [October 1, 2019, 10:53am UTC](https://discuss.elastic.co/t/create-multiple-indexs-with-multiple-input-in-logstash/201779 "2019-10-01T10:53:16Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![Asmaa\_Sarih](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/asmaa_sarih/32/55145_2.png) [@Asmaa\_Sarih](https://discuss.elastic.co/u/Asmaa_Sarih)\
**Post date:** [October 1, 2019, 10:53am UTC](https://discuss.elastic.co/t/create-multiple-indexs-with-multiple-input-in-logstash/201779/1 "2019-10-01T10:53:16Z")

</div>

Hi Team,

Can anyone help me in confugiring multiple indexes from multiple input with logstash,  
I am unable to create multiple index in elastic (with multiple if conditions) . One index is getting created but not both,  
and it works well if I do just one condition with : if , else .  
But if I want to create more than two indexes with multipe input ?,

My logstash.conf file  
input {  
udp {  
host =\> "127.0.0.1"  
port =\> 10514  
codec =\> "json"  
type =\> "rsyslog"  
}  
file {  
type =\> "fortigate"  
path =\> "/home/user/Téléchargements/fortiWebFilter.log"  
sincedb\_path =\> "/dev/null"  
start\_position =\> "beginning"  
}  
}

filter {  
if [type] == "fortigate" {

```auto
grok {
	match => ["message", "%{SYSLOG5424PRI:syslog_index}%{GREEDYDATA:message}"]
	#overwrite => ["message"]
	tag_on_failure => ["failure_grok_fortigate"]
}

kv {

```

value\_split =\> "="

}

mutate {

#I want to use the timestamp inside the logs instead of Logstash's timestamp so we'll first create a new field containing the date and time fields from the syslog before we convert that to the @timestamp field  
add\_field =\> { "temp\_time" =\> "%{date} %{time}" }  
#add\_field =\> { "Desti\_Country" =\> "%{dstip}" }  
#The syslog contains a type field which messes with the Logstash type field so we have to rename it.  
rename =\> { "type" =\> "ftg\_type" }  
#rename =\> { "ip" =\> "Desti\_IP" }  
rename =\> { "subtype" =\> "ftg\_subtype" }  
#add\_field =\> { "type" =\> "forti\_log" }  
convert =\> { "rcvdbyte" =\> "integer" }  
convert =\> { "sentbyte" =\> "integer" }  
}

date {  
match =\> ["temp\_time", "yyyy-MM-dd HH:mm:ss"]  
timezone =\> "UTC"  
target =\> "@timestamp"  
}

geoip {  
source =\> "dstip"  
add\_field =\> ["[geoip][desti\_ip]", "%{[geoip][ip]}" ]  
}

mutate {

#add/remove fields as you see fit.  
remove\_field =\> ["syslog\_index","sessionid","dstcountry","dstip","transip","country\_code3","region\_code","country\_code2","syslog5424\_pri","transport","appcat","srccountry","dstintf","devid","@version","itime","path","logver","logid","vd","host","srcintf","trandisp","location","date","time","service","temp\_time","tags","sentpkt","rcvdpkt","log\_id","message","poluuid"]

remove\_field =\> "[geoip][longitude]"  
remove\_field =\> "[geoip][region\_code]"  
remove\_field =\> "[geoip][country\_code3]"  
remove\_field =\> "[geoip][continent\_code]"  
remove\_field =\> "[geoip][country\_code2]"  
remove\_field =\> "[geoip][latitude]"  
remove\_field =\> "[geoip][location]"  
remove\_field =\> "[geoip][region\_name]"  
remove\_field =\> "[geoip][ip]"  
}

}  
}

output {  
#stdout { codec =\> rubydebug }  
if [type] == "rsyslog" {  
elasticsearch {  
hosts =\> "localhost:9200"  
index =\> "rsyslog-index"  
}  
}  
if [type] == "fortigate"{  
elasticsearch {  
hosts =\> "localhost:9200"  
#http\_compression =\> "true"  
index =\> "forti-index"  
}  
}  
}

---

<div class="post-metadata">

**Author:** ![Justin\_Doles](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/justin_doles/32/40730_2.png) [@Justin\_Doles](https://discuss.elastic.co/u/Justin_Doles)\
**Post date:** [October 1, 2019, 11:06am UTC](https://discuss.elastic.co/t/create-multiple-indexs-with-multiple-input-in-logstash/201779/2 "2019-10-01T11:06:53Z")

</div>

If you strip down your configuration to the bare minimum - does it work?  
input {  
udp {  
host =\> "127.0.0.1"  
port =\> 10514  
codec =\> "json"  
type =\> "rsyslog"  
}  
file {  
type =\> "fortigate"  
path =\> "/home/user/Téléchargements/fortiWebFilter.log"  
sincedb\_path =\> "/dev/null"  
start\_position =\> "beginning"  
}  
}

```
filter {}

output {
    #stdout { codec => rubydebug }
    if [type] == "rsyslog" {
        elasticsearch {
            hosts => ["localhost:9200"]
            index => "rsyslog-index"
        }
    }
    if [type] == "fortigate"{
        elasticsearch {
            hosts => ["localhost:9200"]
            #http_compression => "true"
            index => "forti-index"
        }
    }
}
```

---

<div class="post-metadata">

**Author:** ![Asmaa\_Sarih](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/asmaa_sarih/32/55145_2.png) [@Asmaa\_Sarih](https://discuss.elastic.co/u/Asmaa_Sarih)\
**Post date:** [October 1, 2019, 11:30am UTC](https://discuss.elastic.co/t/create-multiple-indexs-with-multiple-input-in-logstash/201779/3 "2019-10-01T11:30:04Z")

</div>

@Justin_Doles I test that , it's not working

---

<div class="post-metadata">

**Author:** ![Justin\_Doles](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/justin_doles/32/40730_2.png) [@Justin\_Doles](https://discuss.elastic.co/u/Justin_Doles)\
**Post date:** [October 1, 2019, 11:46am UTC](https://discuss.elastic.co/t/create-multiple-indexs-with-multiple-input-in-logstash/201779/4 "2019-10-01T11:46:19Z")

</div>

Are you getting events from each input? What version of Logstash are you running?

I personally do something similar with Logstash. I have multiple inputs that get sent to different indices. But I do not use type (I think I had an issue with type). I use tags on my inputs to sort them. Like this:

```
output {
    if "syslog" in [tags] { elasticsearch {...} }
    if "beats" in [tags] { elasticsearch {....} }
}
```

---

<div class="post-metadata">

**Author:** ![Asmaa\_Sarih](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/asmaa_sarih/32/55145_2.png) [@Asmaa\_Sarih](https://discuss.elastic.co/u/Asmaa_Sarih)\
**Post date:** [October 1, 2019, 11:48am UTC](https://discuss.elastic.co/t/create-multiple-indexs-with-multiple-input-in-logstash/201779/5 "2019-10-01T11:48:49Z")

</div>

@Justin_Doles version of logstash 7.3 ,from udp input ,Ok , I will now test with tags .

I just test simple 2 input , it works ,  
but why not , if I add : filter if [type] == "fortigate" ...

---

<div class="post-metadata">

**Author:** ![Justin\_Doles](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/justin_doles/32/40730_2.png) [@Justin\_Doles](https://discuss.elastic.co/u/Justin_Doles)\
**Post date:** [October 1, 2019, 12:00pm UTC](https://discuss.elastic.co/t/create-multiple-indexs-with-multiple-input-in-logstash/201779/6 "2019-10-01T12:00:23Z")

</div>

I honestly don't know. I just vaguely remember having issues with [type]. Tags work just as well. Same search capabilities.

---

<div class="post-metadata">

**Author:** ![Asmaa\_Sarih](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/asmaa_sarih/32/55145_2.png) [@Asmaa\_Sarih](https://discuss.elastic.co/u/Asmaa_Sarih)\
**Post date:** [October 1, 2019, 12:04pm UTC](https://discuss.elastic.co/t/create-multiple-indexs-with-multiple-input-in-logstash/201779/7 "2019-10-01T12:04:32Z")

</div>

@Justin_Doles can you give me please example with tags how I can use it in input,filter and output

---

<div class="post-metadata">

**Author:** ![Justin\_Doles](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/justin_doles/32/40730_2.png) [@Justin\_Doles](https://discuss.elastic.co/u/Justin_Doles)\
**Post date:** [October 1, 2019, 1:03pm UTC](https://discuss.elastic.co/t/create-multiple-indexs-with-multiple-input-in-logstash/201779/8 "2019-10-01T13:03:10Z")

</div>

Here's a simple example. The only thing you need to remember is the syntax of "if "tag\_name" in [tags] instead of if [type] == "type\_name".

```
input {
    beats {
        port => 5044
        tags => ["beats"]
    }

    udp {
        port => 5514
        source_ip_fieldname => "host.ip"
        tags => ["syslog", "cisco"]
    }
    
    udp {
        port => 5515
        source_ip_fieldname => "host.ip"
        tags => ["syslog", "sonicwall"]
    }
}

filter {
    if "syslog" in [tags] and "sonicwall" in [tags] {
        grok {...}
    }
}

output {
    if "_grokparsefailure" in [tags] {
        file {
            path => "/var/log/logstash/failed_syslog_events-%{+YYYY-MM-dd}.log" 
        }
    }
    if "syslog" in [tags] {
        elasticsearch {
            hosts => ["http://localhost:9200"]
            index => "syslog-1.0"
        }
    }
    if "beats" in [tags] {
        elasticsearch {
            hosts => ["http://localhost:9200"]
            index => "%{[@metadata][beat]}-%{[@metadata][version]}"
        }
    }
}
```

---

<div class="post-metadata">

**Author:** ![Asmaa\_Sarih](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/asmaa_sarih/32/55145_2.png) [@Asmaa\_Sarih](https://discuss.elastic.co/u/Asmaa_Sarih)\
**Post date:** [October 1, 2019, 2:23pm UTC](https://discuss.elastic.co/t/create-multiple-indexs-with-multiple-input-in-logstash/201779/9 "2019-10-01T14:23:56Z")

</div>

Thank you , @Justin_Doles please have you any idea how to configure Fortigate firewall to send logs to Logstash

---

<div class="post-metadata">

**Author:** ![Justin\_Doles](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/justin_doles/32/40730_2.png) [@Justin\_Doles](https://discuss.elastic.co/u/Justin_Doles)\
**Post date:** [October 1, 2019, 2:35pm UTC](https://discuss.elastic.co/t/create-multiple-indexs-with-multiple-input-in-logstash/201779/10 "2019-10-01T14:35:34Z")

</div>

Assuming Fortigate supports syslog - I'd set up another UDP input in Logstash to handle it. Just set the port to a different number and tag it appropriately. You can see how I do that in the example. If it's plain syslog, you may even be able to use Logstash's syslog input. I haven't had much luck with that myself.

---

<div class="post-metadata">

**Author:** ![Asmaa\_Sarih](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/asmaa_sarih/32/55145_2.png) [@Asmaa\_Sarih](https://discuss.elastic.co/u/Asmaa_Sarih)\
**Post date:** [October 1, 2019, 2:57pm UTC](https://discuss.elastic.co/t/create-multiple-indexs-with-multiple-input-in-logstash/201779/11 "2019-10-01T14:57:58Z")

</div>

@Justin_Doles , I install elk stack (elasticsearch, logstash and kibana) in my local machine , when the log data augments my machine is blocked how to manage it ? or where deploy elasticsearch data

---

<div class="post-metadata">

**Author:** ![Justin\_Doles](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/justin_doles/32/40730_2.png) [@Justin\_Doles](https://discuss.elastic.co/u/Justin_Doles)\
**Post date:** [October 1, 2019, 5:39pm UTC](https://discuss.elastic.co/t/create-multiple-indexs-with-multiple-input-in-logstash/201779/12 "2019-10-01T17:39:56Z")

</div>

@Asmaa_Sarih I'm not sure what you're asking. The installation of Elasticsearch is dependent on the OS you are using. [https://www.elastic.co/guide/en/elasticsearch/reference/current/install-elasticsearch.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/install-elasticsearch.html)  
You can control where the data lives by editing the elasticsearch.yml file. [https://www.elastic.co/guide/en/elasticsearch/reference/current/path-settings.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/path-settings.html)

---

<div class="post-metadata">

**Author:** ![Asmaa\_Sarih](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/asmaa_sarih/32/55145_2.png) [@Asmaa\_Sarih](https://discuss.elastic.co/u/Asmaa_Sarih)\
**Post date:** [October 2, 2019, 9:21am UTC](https://discuss.elastic.co/t/create-multiple-indexs-with-multiple-input-in-logstash/201779/13 "2019-10-02T09:21:41Z")

</div>

@Justin_Doles How I can get logs from remote machine

---

<div class="post-metadata">

**Author:** ![Justin\_Doles](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/justin_doles/32/40730_2.png) [@Justin\_Doles](https://discuss.elastic.co/u/Justin_Doles)\
**Post date:** [October 2, 2019, 11:52am UTC](https://discuss.elastic.co/t/create-multiple-indexs-with-multiple-input-in-logstash/201779/14 "2019-10-02T11:52:33Z")

</div>

I'm not familiar with Fortigate, but you would set a syslog destination as the IP and port that you have configured in Logstash. [https://help.fortinet.com/fadc/4-5-1/olh/Content/FortiADC/handbook/log\_remote.htm](https://help.fortinet.com/fadc/4-5-1/olh/Content/FortiADC/handbook/log_remote.htm)

In the example I gave, I'm using UDP port 5515. I personally like to test just using the stdout output plugin. It writes the log entries to the console - assuming you're launching Logststash from the console. [https://www.elastic.co/guide/en/logstash/current/plugins-outputs-stdout.html](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-stdout.html)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 30, 2019, 11:52am UTC](https://discuss.elastic.co/t/create-multiple-indexs-with-multiple-input-in-logstash/201779/15 "2019-10-30T11:52:37Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.