# Logstash-Forwarder and Redis

**URL:** <https://discuss.elastic.co/t/logstash-forwarder-and-redis/2702>\
**Category:** Logstash\
**Created:** [June 15, 2015, 2:26pm UTC](https://discuss.elastic.co/t/logstash-forwarder-and-redis/2702 "2015-06-15T14:26:56Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![bdunbar](https://avatars.discourse-cdn.com/v4/letter/b/a3d4f5/32.png) [@bdunbar](https://discuss.elastic.co/u/bdunbar)\
**Post date:** [June 15, 2015, 2:26pm UTC](https://discuss.elastic.co/t/logstash-forwarder-and-redis/2702/1 "2015-06-15T14:26:56Z")

</div>

I want to try using Redis with logstash-forwarder, but I'm confused on a point.

When forwarder tires to connect it complains about a tls handshake and fails to connect.

Can forwarder connect to redis? [This post](https://ianunruh.com/2014/05/monitor-everything-part-2.html) suggests that for 'forwarder' to Redis the full-on logstash is needed on the client.

As always, thanks in advance.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [June 15, 2015, 10:59pm UTC](https://discuss.elastic.co/t/logstash-forwarder-and-redis/2702/2 "2015-06-15T22:59:14Z")

</div>

LSF can only talk to LS, it cannot interact with redis.  
See [here](https://github.com/elastic/logstash-forwarder#logstash-forwarder) for the docs.

---

<div class="post-metadata">

**Author:** ![sunilmchaudhari](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sunilmchaudhari/32/9475_2.png) [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\
**Post date:** [July 3, 2015, 7:05am UTC](https://discuss.elastic.co/t/logstash-forwarder-and-redis/2702/3 "2015-07-03T07:05:58Z")

</div>

Hello Mark,  
I am going through the same problem.  
I have server where LS is running in shipper and Indexer mode with redis in between.  
Shipper--\> input= lumberjack and output =redis  
Indexer--\> input = redis and output = elasticsearch  
LSF is running on client machines which talks to LS shipper.

I want LSF to talk with redis so that I can store some log file specific keys and then create those many indexes on the basis of keys (e.g keyName-YYYY-MM-DD)  
But As you said LSF cant talk to redis then I have to bring shipper to client side.  
So what is benefit of LSF?  
How can I create multiple indexes in ES through one LS indexer running on server? sample configuration please?

thanks  
Sunil Chaudhari

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [July 3, 2015, 7:33am UTC](https://discuss.elastic.co/t/logstash-forwarder-and-redis/2702/4 "2015-07-03T07:33:23Z")

</div>

I think [LSF's README file](https://github.com/elastic/logstash-forwarder/blob/master/README.md) explains it's reason for being. See below for how to create different indexing depending on the message.

> [@How to handle multiple inputs with Logstash to different indices](https://discuss.elastic.co/t/how-to-handle-multiple-inputs-with-logstash-to-different-indices/24541/2):
>
> Use the [grok filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html) to extract the project name from the input file path (stored in the path field), then reference that field when setting the index pattern of the [elasticsearch output](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html). elasticsearch { ... index =\> "logstash-%{project}-%{+YYYY.MM.dd}" }

---

<div class="post-metadata">

**Author:** ![sunilmchaudhari](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sunilmchaudhari/32/9475_2.png) [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\
**Post date:** [July 3, 2015, 9:08am UTC](https://discuss.elastic.co/t/logstash-forwarder-and-redis/2702/5 "2015-07-03T09:08:39Z")

</div>

Hi Marko,  
Thanks for quick reply.

In the given example, you are creating index on the basis of project name which you take from path.  
I want to create indexes from values of 'Application' field which I set in logstash-forwarder on client.  
For example:  
in **LSF config on CRM client 1** I say  
{  
"paths": [  
"/var/log/crm/crmERROR.log"  
],  
"fields": {" **Application":"CRM**","Sub-System":"Mysystem", "type":"Error-logs "  
}

in **LSF config on sales client 2** I say  
{  
"paths": [  
"/var/log/sales/SalesERROR.log"  
],  
"fields": {" **Application":"Sales**","Sub-System":"Myststem", "type":"Error-logs "  
}

all those logs will be sent to redis through shipper and indexer will pick it up from redis.  
Now in LS indexer I want to create indexes on the basis of this field "Application", How Can I get this value of Application field so that I can use it in output to create indexname.

br,  
Sunil.

---

<div class="post-metadata">

**Author:** ![sunilmchaudhari](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sunilmchaudhari/32/9475_2.png) [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\
**Post date:** [July 3, 2015, 9:40am UTC](https://discuss.elastic.co/t/logstash-forwarder-and-redis/2702/6 "2015-07-03T09:40:13Z")

</div>

Sorry for incorrect name in salutation part.  
Please read it as "Magnus"

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [July 3, 2015, 10:46am UTC](https://discuss.elastic.co/t/logstash-forwarder-and-redis/2702/7 "2015-07-03T10:46:21Z")

</div>

The answer I linked to contains the answer to your question. You can refer to you Application field in exactly the same way.

But let's not hijack the original topic with a completely different question. Please start a new topic if you have follow-up questions.

(Also, note that you can edit your own post if you get something wrong. That's preferably to posting new messages with corrections.)

---

<div class="post-metadata">

**Author:** ![sunilmchaudhari](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sunilmchaudhari/32/9475_2.png) [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\
**Post date:** [July 3, 2015, 10:58am UTC](https://discuss.elastic.co/t/logstash-forwarder-and-redis/2702/8 "2015-07-03T10:58:27Z")

</div>

Thank you very much Magnus.  
Will use it as directed.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:35am UTC](https://discuss.elastic.co/t/logstash-forwarder-and-redis/2702/9 "2017-07-06T05:35:35Z")

</div>


