# Logstash config with redis

**URL:** <https://discuss.elastic.co/t/logstash-config-with-redis/126424>\
**Category:** Logstash\
**Created:** [April 2, 2018, 12:38pm UTC](https://discuss.elastic.co/t/logstash-config-with-redis/126424 "2018-04-02T12:38:34Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Gaurav\_Agarwal](https://avatars.discourse-cdn.com/v4/letter/g/9fc29f/32.png) [@Gaurav\_Agarwal](https://discuss.elastic.co/u/Gaurav_Agarwal)\
**Post date:** [April 2, 2018, 12:38pm UTC](https://discuss.elastic.co/t/logstash-config-with-redis/126424/1 "2018-04-02T12:38:34Z")

</div>

While using redis as the in-memory database the logstash config file is not working fine whereas when in place redis if I use the file input plugin it does not fail at all. I am not able to understand what is wrong with my config file where I use redis as the input plugin.

Redis forwader  
input {  
file {  
path =\> "/logs/app\__/_.log"  
exclude =\> [  
"access\*.log\*",  
"fe\*.log\*",  
"stat\*.log\*",  
"gc\*.log\*",  
"_dump_.log\*"  
]  
sincedb\_path =\> "/dev/null"  
type =\> "app"  
codec =\> multiline {  
pattern =\> "^%{YEAR}/%{MONTHNUM}/%{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND}"  
negate =\> true  
what =\> "previous"  
}  
}  
}

filter {

```
grok {
	match => {
		"path" => "/logs/(?<app>[^/]+)/[^/]+/logs/(?<instance>[^/]+)/[^/]+"
	}
	overwrite => ["host"]
}

```

}

output {

# stdout { codec =\> rubydebug }

```
redis {
	host => "gauapt13"
	data_type => "list"
	key => "logstash-app"
}

```

}

redis indexer  
input {  
redis {  
host =\> "gauapt13"  
data\_type =\> "list"  
key =\> "logstash-app"  
codec =\> json  
threads =\> 2

codec =\> multiline {  
pattern =\> "^%{YEAR}/%{MONTHNUM}/%{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND}"  
negate =\> true  
what =\> "previous"  
}

}  
}  
filter {

```
if [type] == "app" {
	grok {
		match => {
			# "message" => "%{APT_TIMESTAMP:timestamp} \| %{HOSTNAME:hostname} \| %{DATA:application} \| %{APT_LOGLEVEL:loglevel} *\| 1-%{DATA:thread:int} \| %{DATA:class} *\| %{DATA:correlationId} \| %{GREEDYDATA:msg}"
			"message" => "%{APT_TIMESTAMP:timestamp} \| \S+ \| \S+ \| %{APT_LOGLEVEL:loglevel} *\| 1-%{DATA:thread:int} \| %{DATA:class} *\| %{DATA:correlationId} \| %{GREEDYDATA:msg}"
		}
		patterns_dir => "/logs/pattern"
		remove_field => ["message"]
	}
}

if "_grokparsefailure" not in [tags] {
    date {
        match => ["timestamp", "YYYY/MM/dd HH:mm:ss.SSS", "YYYY/MM/dd HH:mm:ss,SSS", "dd/MMM/YYYY:HH:mm:ss +0000", "EEE MMM dd HH:mm:ss YYYY"]
        timezone => "UTC"
	}
	if [type] == "web_access" or [type] == "web_error" {
        mutate {
	        gsub => [
                "referrer","\"","",
                "agent","\"","",
                "JSESSIONID","\"","",
                "APT_SESSIONID","\"","",
                "correlationId","\"","",
                "transactionOriginator","\"","",
                "customerId","\"",""
            ]
	        remove_field => ["logline", "timestamp", "BASE10NUM", "INT", "HOSTNAME", "IPV4", "day", "month", "monthday", "time", "year"]
		}
	}

}

```

}

output {  
stdout{ codec =\> rubydebug}  
if [app] == "aaa" or [app] == "amga" or [type] == "fem" or [type] == "sepal" or [type] == "web\_access" {  
elasticsearch {  
hosts =\> "gauapt13"  
index =\> "apt-%{+YYYY.MM.dd}"

```
	}
}
else {
	elasticsearch {
		hosts => "gauapt13"
		index => "apt-%{+YYYY.MM.dd}"
        
}

	
        

}

```

}

---

<div class="post-metadata">

**Author:** ![rcowart](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rcowart/32/88091_2.png) [@rcowart](https://discuss.elastic.co/u/rcowart)\
**Post date:** [April 3, 2018, 8:42am UTC](https://discuss.elastic.co/t/logstash-config-with-redis/126424/2 "2018-04-03T08:42:35Z")

</div>

Can you please use markdown to format the code in your post so it is not so difficult to read? This will make it easier for someone to help.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 1, 2018, 8:42am UTC](https://discuss.elastic.co/t/logstash-config-with-redis/126424/3 "2018-05-01T08:42:39Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.