# Logs not being indexed in Elasticsearch from Redis
**URL:** https://discuss.elastic.co/t/logs-not-being-indexed-in-elasticsearch-from-redis/189489
**Category:** Logstash
**Created:** [July 9, 2019, 8:40am UTC](https://discuss.elastic.co/t/logs-not-being-indexed-in-elasticsearch-from-redis/189489 "2019-07-09T08:40:52Z")
**Posts on this page:** 4
**Page:** 1
### Author:  [@proudboffin](https://discuss.elastic.co/u/proudboffin)
#### Post date: [July 9, 2019, 8:40am UTC](https://discuss.elastic.co/t/logs-not-being-indexed-in-elasticsearch-from-redis/189489/1 "2019-07-09T08:40:52Z")
Hi all,
Trying to build the following pipeline for apache access logs: Filebeat -\> Redis -\> Logstash -\> Elasticsearch.
All components are configured (see below configs) and start successfully: Filebeat publishes the logs to an "apache" channel on Redis and Logstash subscribes to the channel. A Logstash index is created on Elasticsearch but it remains empty of documents. Logstash, Elasticsearch and Redis logs contain no errors.
**Configurations:**
Filebeat:
```
filebeat.inputs:
- type: log
enabled: true
paths:
- /var/log/apache2/access.log
output.redis:
hosts: ["localhost"]
key: "apache"
db: 0
timeout: 5
```
Logstash:
```
input {
redis {
host => "localhost"
key => "apache"
data_type => "channel"
codec => json
}
}
filter {
grok {
match => { "message" => "%{COMBINEDAPACHELOG}" }
}
date {
match => ["timestamp" , "dd/MMM/yyyy:HH:mm:ss Z"]
}
geoip {
source => "clientip"
}
}
output {
elasticsearch {
hosts => ["localhost:9200"]
}
}
```
An example of a messages in the Redis channel:
`1562657837.663939 [0 127.0.0.1:33972] "RPUSH" "apache" "{\"@timestamp\":\"2019-07-09T07:37:16.662Z\",\"@metadata\":{\"beat\":\"filebeat\",\"type\":\"_doc\",\"version\":\"7.2.0\"},\"log\":{\"offset\":1984,\"file\":{\"path\":\"/var/log/apache2/access.log\"}},\"message\":\"91.205.154.22 - - [09/Jul/2019:07:37:15 +0000] \\\"GET /hello.html HTTP/1.1\\\" 304 180 \\\"-\\\" \\\"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36\\\"\",\"input\":{\"type\":\"log\"},\"agent\":{\"id\":\"736b2ac9-9062-4705-9405-f2233250a82e\",\"version\":\"7.2.0\",\"type\":\"filebeat\",\"ephemeral_id\":\"d000dc8d-83f3-4975-83d0-db7f85c2167e\",\"hostname\":\"ip-172-31-26-146\"},\"ecs\":{\"version\":\"1.0.0\"},\"host\":{\"name\":\"ip-172-31-26-146\"}}"`
Thanks!
---
### Author:  [@guyboertje](https://discuss.elastic.co/u/guyboertje)
#### Post date: [July 9, 2019, 4:40pm UTC](https://discuss.elastic.co/t/logs-not-being-indexed-in-elasticsearch-from-redis/189489/2 "2019-07-09T16:40:21Z")
Beats is pushing to the tail of a LIST and, with `data_type => "channel"`, Logstash is subscribing to a CHANNEL - 2 different data structures in Redis. You need `data_type => "list"`.
---
### Author:  [@proudboffin](https://discuss.elastic.co/u/proudboffin)
#### Post date: [July 9, 2019, 6:14pm UTC](https://discuss.elastic.co/t/logs-not-being-indexed-in-elasticsearch-from-redis/189489/3 "2019-07-09T18:14:11Z")
> [@guyboertje](#):
>
> data\_type =\> "list"
Thank you! Yes, that seems to have been the issue. Added `data_type: "list"` to the Filebeat config and defined the same value for the same setting in Logstash as well. That works now.
Oddly though, setting the value to `channel` in both Filebeat and Logstash doesn't seem to work, which would be my preferred method to use Redis Pub/Sub. Any idea why?
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 6, 2019, 6:14pm UTC](https://discuss.elastic.co/t/logs-not-being-indexed-in-elasticsearch-from-redis/189489/4 "2019-08-06T18:14:12Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.