# Logstash Shipper to Redis **URL:** \ **Category:** Logstash\ **Created:** [September 24, 2015, 10:17pm UTC](https://discuss.elastic.co/t/logstash-shipper-to-redis/29948 "2015-09-24T22:17:03Z")\ **Posts on this page:** 17\ **Page:** 1
**Author:** ![ariceELK](https://avatars.discourse-cdn.com/v4/letter/a/ba9def/32.png) [@ariceELK](https://discuss.elastic.co/u/ariceELK)\ **Post date:** [September 24, 2015, 10:17pm UTC](https://discuss.elastic.co/t/logstash-shipper-to-redis/29948/1 "2015-09-24T22:17:03Z")
Hi, Im using LS Shipper to send to Redis queue. The conf input is reading from different log files but I only see logs from type syslog being sent. Application and Web logs are not getting sent. I've run LS shipper in debug mode and see it has found all the logs with no errors on startup, config looks good. Anything else to check? Also is there a way to encrypt the traffic from the LS shipper and Redis ? Thanks Aidan ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [September 25, 2015, 5:45am UTC](https://discuss.elastic.co/t/logstash-shipper-to-redis/29948/2 "2015-09-25T05:45:08Z")
> The conf input is reading from different log files but I only see logs from type syslog being sent. Application and Web logs are not getting sent. > I've run LS shipper in debug mode and see it has found all the logs with no errors on startup, config looks good. And new data is being added to the application and web logs? > Also is there a way to encrypt the traffic from the LS shipper and Redis ? I don't believe Redis supports encryption. You could tunnel the traffic over e.g. SSH, or have the Logstash shippers send via the lumberjack protocol to another Logstash instance on the Redis box, which then passes events to Redis on localhost. ---
**Author:** ![ariceELK](https://avatars.discourse-cdn.com/v4/letter/a/ba9def/32.png) [@ariceELK](https://discuss.elastic.co/u/ariceELK)\ **Post date:** [September 25, 2015, 10:00am UTC](https://discuss.elastic.co/t/logstash-shipper-to-redis/29948/3 "2015-09-25T10:00:17Z")
I added: sincedb\_path =\> "/dev/null" start\_position =\> "beginning" and it reads the files now. Thank you. ---
**Author:** ![ariceELK](https://avatars.discourse-cdn.com/v4/letter/a/ba9def/32.png) [@ariceELK](https://discuss.elastic.co/u/ariceELK)\ **Post date:** [September 25, 2015, 10:25am UTC](https://discuss.elastic.co/t/logstash-shipper-to-redis/29948/4 "2015-09-25T10:25:40Z")
What is the preferred way to set logstash permission on the files as getting an error that was working yesterday? failed to open /var/log/messages: Permission denied - /var/log/messages {:level=\>:warn, :file=\>"filewatch/tail.rb", :line=\>"131", :method=\>"\_open\_file"} ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [September 25, 2015, 11:01am UTC](https://discuss.elastic.co/t/logstash-shipper-to-redis/29948/5 "2015-09-25T11:01:04Z")
What does `ls -l /var/log/messages` say? ---
**Author:** ![ariceELK](https://avatars.discourse-cdn.com/v4/letter/a/ba9def/32.png) [@ariceELK](https://discuss.elastic.co/u/ariceELK)\ **Post date:** [September 25, 2015, 11:40am UTC](https://discuss.elastic.co/t/logstash-shipper-to-redis/29948/6 "2015-09-25T11:40:40Z")
-rw-------+ 1 root root 650898854 Sep 25 04:31 /var/log/messages I set setfacl -m u:logstash:r /var/log/messages yesterday. ![](https://us1.discourse-cdn.com/elastic/original/2X/c/cb6477d7486e12f28258ffdc8751d356bee647a8.jpg) ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [September 25, 2015, 11:47am UTC](https://discuss.elastic.co/t/logstash-shipper-to-redis/29948/7 "2015-09-25T11:47:38Z")
Perhaps the file has rotated since then? If so you should be able to add setfacl as a post rotation command. I'd also look into the possibility of configuring what permissions these files should have. Which Linux is this? On Debian and Ubuntu the log files are mode 0640 with "adm" as the group, so adding Logstash to that group solves the problem. ---
**Author:** ![ariceELK](https://avatars.discourse-cdn.com/v4/letter/a/ba9def/32.png) [@ariceELK](https://discuss.elastic.co/u/ariceELK)\ **Post date:** [September 25, 2015, 1:50pm UTC](https://discuss.elastic.co/t/logstash-shipper-to-redis/29948/8 "2015-09-25T13:50:20Z")
Its CentOS. Does adding an entry in /etc/sysconfig/logstash LS\_GROUP=adm work ? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [September 25, 2015, 2:03pm UTC](https://discuss.elastic.co/t/logstash-shipper-to-redis/29948/9 "2015-09-25T14:03:26Z")
Only if the members of the adm group have permission to read the log file. I don't know of a way to configure the permissions of these log files so your best bet might be a post rotation script that adjusts the ACL or the file mode. Syslog might not open all files immediately, and in that case you may have to make sure that the file is created in the post rotation script so that you can change its permissions. ---
**Author:** ![ariceELK](https://avatars.discourse-cdn.com/v4/letter/a/ba9def/32.png) [@ariceELK](https://discuss.elastic.co/u/ariceELK)\ **Post date:** [October 24, 2015, 11:51am UTC](https://discuss.elastic.co/t/logstash-shipper-to-redis/29948/10 "2015-10-24T11:51:15Z")
Hi Magnus, Thanks. We will make the necessary changes outside of LS.. I have a question regarding monitoring LS. Are there any planned API’s to allow us to do this? Can you confirm that Marvel does not need a licence anymore ? Many thanks Aidan ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [October 25, 2015, 9:30am UTC](https://discuss.elastic.co/t/logstash-shipper-to-redis/29948/11 "2015-10-25T09:30:56Z")
> I have a question regarding monitoring LS. Are there any planned API’s to allow us to do this? Yes, see the [Logstash roadmap](https://www.elastic.co/guide/en/logstash-roadmap/current/index.html). > Can you confirm that Marvel does not need a licence anymore ? No, I haven't heard anything about that. ---
**Author:** ![ariceELK](https://avatars.discourse-cdn.com/v4/letter/a/ba9def/32.png) [@ariceELK](https://discuss.elastic.co/u/ariceELK)\ **Post date:** [October 30, 2015, 2:39pm UTC](https://discuss.elastic.co/t/logstash-shipper-to-redis/29948/12 "2015-10-30T14:39:50Z")
Regarding Marvel, this is conflicting with what we were told at a training day in Dublin recently. The trainers said that Marvel or Marvel-lite would be available to the community ? ![](https://us1.discourse-cdn.com/elastic/original/2X/c/cb6477d7486e12f28258ffdc8751d356bee647a8.jpg) ---
**Author:** ![ariceELK](https://avatars.discourse-cdn.com/v4/letter/a/ba9def/32.png) [@ariceELK](https://discuss.elastic.co/u/ariceELK)\ **Post date:** [December 1, 2015, 12:39pm UTC](https://discuss.elastic.co/t/logstash-shipper-to-redis/29948/13 "2015-12-01T12:39:25Z")
Hi Magnus, Is there any benchmarks on disk i/o that Elasticsearch uses as compared to Graphite ? Many thanks Aidan ![](https://us1.discourse-cdn.com/elastic/original/2X/c/cc3f90e2d51d14117455e0000759b4086072762c.jpg) ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [December 1, 2015, 12:59pm UTC](https://discuss.elastic.co/t/logstash-shipper-to-redis/29948/14 "2015-12-01T12:59:25Z")
> Is there any benchmarks on disk i/o that Elasticsearch uses as compared to Graphite ? None that I'm aware of, but my hunch is that Graphite is friendlier I/O-wise. ---
**Author:** ![ariceELK](https://avatars.discourse-cdn.com/v4/letter/a/ba9def/32.png) [@ariceELK](https://discuss.elastic.co/u/ariceELK)\ **Post date:** [December 18, 2015, 4:20pm UTC](https://discuss.elastic.co/t/logstash-shipper-to-redis/29948/15 "2015-12-18T16:20:10Z")
Hi Magnus, I have collectd sending metrics to Logstash and want to create graphs form memory usage, etc. Any docs for this / Many thanks Aidan ![](https://us1.discourse-cdn.com/elastic/original/2X/c/cc3f90e2d51d14117455e0000759b4086072762c.jpg) ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [January 2, 2016, 7:31pm UTC](https://discuss.elastic.co/t/logstash-shipper-to-redis/29948/16 "2016-01-02T19:31:52Z")
That's a very open question. Have you looked at the Kibana introduction videos and documentation? (I'm assuming you're sending the data to Elasticsearch and want to use Kibana to plot it.) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 5:17am UTC](https://discuss.elastic.co/t/logstash-shipper-to-redis/29948/17 "2017-07-06T05:17:03Z")