# Date plugin lose the stream to ElasticSearch

**URL:** <https://discuss.elastic.co/t/date-plugin-lose-the-stream-to-elasticsearch/129906>\
**Category:** Logstash\
**Created:** [April 28, 2018, 7:44am UTC](https://discuss.elastic.co/t/date-plugin-lose-the-stream-to-elasticsearch/129906 "2018-04-28T07:44:35Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![liuzhen](https://avatars.discourse-cdn.com/v4/letter/l/bbce88/32.png) [@liuzhen](https://discuss.elastic.co/u/liuzhen)\
**Post date:** [April 28, 2018, 7:44am UTC](https://discuss.elastic.co/t/date-plugin-lose-the-stream-to-elasticsearch/129906/1 "2018-04-28T07:44:35Z")

</div>

Hi,

I am setting up one port for all my beats input, including redis & haproxy logs, where haproxy logs work but redis logs do not get into elastic search (after adding Date plugin).

if I remove the Data plugin in the redis stream, it works (redis logs get into ES);  
it does not append any dateparsefailure tag, so I assume the parse is ok;

even I turned on config.debug = true and log.level = debug, did not see any error or exception;

I have tried to rename the "timestamp" to "timestamp\_redis" in redis stream, but it did not help.

Any suggestion?

## config.yml:

```
 input {
        beats {
                port => 5001
        }
}

## Add your filters / logstash plugins configuration here

filter {
        if [fileset][module] == "redis" {
                grok {
                        match => { "message" => ["(\[)?%{POSINT:pid}(\])?(:)?%{NOTSPACE:[redis][log][role]} %{REDISTIMESTAMP:timestamp} %{NOTSPACE:[redis][log][level]} %{GREEDYDATA:what}"] }
                }
                date {
                        match => ["timestamp", "dd MMM HH:mm:ss.SSS"]
                }
        }
        else if [source] == "/opt/logs/haproxy/haproxy.log" {
                grok {
                        match => { "message" => ["%{HAPROXYTCP}"] }
                }
                date {
                        match => ["timestamp8601", "ISO8601"]
                }
        }
}

output {
        elasticsearch {
                hosts => "elasticsearch:9200"
                index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
        }
        stdout {
                codec => "rubydebug"
        }
    }

```

## rubydebug:

```
{
     "timestamp" => "28 Apr 15:14:38.239",
        "source" => "/opt/data/redis_cluster/logs/5162.1060",
          "tags" => [
        [0] "beats_input_codec_plain_applied"
    ],
        "offset" => 249810163,
          "what" => "DB 0: 4725 keys (4395 volatile) in 8192 slots HT.",
           "pid" => "124455",
    "@timestamp" => 2018-04-28T15:14:38.239Z,
    "prospector" => {
        "type" => "log"
    },
      "@version" => "1",
          "beat" => {
            "name" => "yz-8-111",
         "version" => "6.2.2",
        "hostname" => "yz-8-111"
    },
       "fileset" => {
          "name" => "log",
        "module" => "redis"
    },
         "redis" => {
        "log" => {
            "level" => "-",
             "role" => "M"
        }
    },
          "host" => "yz-8-111",
       "message" => "124455:M 28 Apr 15:14:38.239 - DB 0: 4725 keys (4395 volatile) in 8192 slots HT."
}
```

---

<div class="post-metadata">

**Author:** ![liuzhen](https://avatars.discourse-cdn.com/v4/letter/l/bbce88/32.png) [@liuzhen](https://discuss.elastic.co/u/liuzhen)\
**Post date:** [April 28, 2018, 8:12am UTC](https://discuss.elastic.co/t/date-plugin-lose-the-stream-to-elasticsearch/129906/2 "2018-04-28T08:12:40Z")

</div>

Ah, I got it.  
I am in +8 timezone, and the redis timestamp has no timezone pre-set, so the logs output to the future.

query the future (+8:00) confirmed my gotcha.

thanks

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 26, 2018, 8:12am UTC](https://discuss.elastic.co/t/date-plugin-lose-the-stream-to-elasticsearch/129906/3 "2018-05-26T08:12:44Z")

</div>

Closed on May 26, 2018, 8:12am UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.