# Quick question. Logstash Multiple output to ES

**URL:** <https://discuss.elastic.co/t/quick-question-logstash-multiple-output-to-es/57521>\
**Category:** Logstash\
**Created:** [August 8, 2016, 7:15pm UTC](https://discuss.elastic.co/t/quick-question-logstash-multiple-output-to-es/57521 "2016-08-08T19:15:39Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![gh0stid](https://avatars.discourse-cdn.com/v4/letter/g/f9ae1b/32.png) [@gh0stid](https://discuss.elastic.co/u/gh0stid)\
**Post date:** [August 8, 2016, 7:15pm UTC](https://discuss.elastic.co/t/quick-question-logstash-multiple-output-to-es/57521/1 "2016-08-08T19:15:39Z")

</div>

Hello, once again, its me !

( yeah .. i know .. )

anyway!

I would like to know how would someone succeed in re-indexing to a second index only part of the "logs" that match XY filter.

lemme give you an exemple...  
I currentely index everything coming in my network.log file wich are fortigate logs, for now.

We woud like to be able to "fetch" only those whose dstip match local subnet AND reindex em in a new "index" named "ritm-\*" that would permit me, to simply run a crontab and delete that specific index each day and restart over ( to save space ) while, having the whole data on my other index. for future investigation.

long story made short, heres my config so far :

apprentely, this, doesnt work. dont know why .. could you guys give me a quick hand please ?

Thank you !

---

<div class="post-metadata">

**Author:** ![gh0stid](https://avatars.discourse-cdn.com/v4/letter/g/f9ae1b/32.png) [@gh0stid](https://discuss.elastic.co/u/gh0stid)\
**Post date:** [August 9, 2016, 1:27pm UTC](https://discuss.elastic.co/t/quick-question-logstash-multiple-output-to-es/57521/2 "2016-08-09T13:27:38Z")

</div>

heres the error :

/opt/logstash/bin/logstash -f /etc/logstash/conf.d/10-network\_log.conf --configtest

> The given configuration is invalid. Reason: Expected one of #, { at line 20, column 9 (byte 291) after filter{

> grok {  
> match =\> [  
> "message",  
> "%{TIMESTAMP\_ISO8601:logtimestamp} %{GREEDYDATA:kv}"  
> ]  
> remove\_field =\> ["message"]  
> }

> if ([dstip] =~ /^10./) {  
> add\_tag {:level=\>:fatal}

and again , the config

cat 10-network\_log.conf

> input {  
> file {  
> path =\> ["/var/log/network.log"]  
> start\_position =\> "beginning"  
> type =\> "FTG"  
> }  
> }

> filter{

> grok {  
> match =\> [  
> "message",  
> "%{TIMESTAMP\_ISO8601:logtimestamp} %{GREEDYDATA:kv}"  
> ]  
> remove\_field =\> ["message"]  
> }

> if ([dstip] =~ /^10./) {  
> add\_tag =\> "traffic\_lan"  
> }

> kv {  
> source =\> "kv"  
> field\_split =\> " "  
> value\_split =\> "="  
> }

> date {  
> match =\> ["logtimestamp", "ISO8601"]  
> locale =\> "en"  
> remove\_field =\> ["logtimestamp"]  
> }

> mutate {  
> convert =\> ["rcvdbyte", "integer"]  
> convert =\> ["countdlp", "integer"]  
> convert =\> ["countweb", "integer"]  
> convert =\> ["countav", "integer"]  
> convert =\> ["countemail", "integer"]  
> convert =\> ["countips", "integer"]  
> convert =\> ["duration", "integer"]  
> convert =\> ["sentpkt", "integer"]  
> convert =\> ["rcvdpkt", "integer"]  
> convert =\> ["sentbyte", "integer"]  
> convert =\> ["shaperdroprcvdbyte", "integer"]  
> convert =\> ["shaperdropsentbyte", "integer"]  
> convert =\> ["filesize", "integer"]  
> convert =\> ["count", "integer"]  
> convert =\> ["total", "integer"]  
> convert =\> ["totalsession", "integer"]  
> convert =\> ["bandwidth", "integer"]  
> add\_tag =\> "fortigate\_log"

> }

> geoip{  
> source =\>"dstip"  
> database =\>"/opt/logstash/GeoLiteCity.dat"  
> }

> geoip{  
> source =\>"srcip"  
> database =\>"/opt/logstash/GeoLiteCity.dat"  
> }

> }

cat 50-elasticsearch-output.conf

> output {  
> if "fortigate\_log" in [tags] {  
> elasticsearch {  
> hosts =\> ["localhost:9200"]  
> index =\> ["ftg-%{+YYYY.MM.dd}"]  
> }  
> }

> else if "traffic\_lan" in [tags] {  
> elasticsearch {  
> hosts =\> ["localhost:9200"]  
> index =\> ["ritm-%{+YYYY.MM.dd}"]  
> }

> }

> else {  
> file {  
> path =\> "/var/log/logstash/unknown\_messages.log"  
> }

> }  
> }

somebody could help me please ?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 10, 2016, 5:53am UTC](https://discuss.elastic.co/t/quick-question-logstash-multiple-output-to-es/57521/3 "2016-08-10T05:53:58Z")

</div>

> if ([dstip] =~ /^10./) {  
> add\_tag =\> "traffic\_lan"  
> }

`add_tag` needs to be inside a filter. Put it in a mutate filter, for example.

---

<div class="post-metadata">

**Author:** ![gh0stid](https://avatars.discourse-cdn.com/v4/letter/g/f9ae1b/32.png) [@gh0stid](https://discuss.elastic.co/u/gh0stid)\
**Post date:** [August 10, 2016, 4:33pm UTC](https://discuss.elastic.co/t/quick-question-logstash-multiple-output-to-es/57521/4 "2016-08-10T16:33:48Z")

</div>

Thank you.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:43am UTC](https://discuss.elastic.co/t/quick-question-logstash-multiple-output-to-es/57521/5 "2017-07-06T04:43:57Z")

</div>