# Filter issue with logstash beta 2.0

**URL:** <https://discuss.elastic.co/t/filter-issue-with-logstash-beta-2-0/32662>\
**Category:** Logstash\
**Created:** [October 21, 2015, 11:03am UTC](https://discuss.elastic.co/t/filter-issue-with-logstash-beta-2-0/32662 "2015-10-21T11:03:42Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![boeriksencom](https://avatars.discourse-cdn.com/v4/letter/b/e274bd/32.png) [@boeriksencom](https://discuss.elastic.co/u/boeriksencom)\
**Post date:** [October 21, 2015, 11:03am UTC](https://discuss.elastic.co/t/filter-issue-with-logstash-beta-2-0/32662/1 "2015-10-21T11:03:42Z")

</div>

Hi, I've just installed elaticsearch 2.0.0-rc1, kibana 4.2.0-beta2 and logstash 2.0.0-beta2.

I have a logstash config that picks up data from at txt file:

Textfile example data:  
1;127.0.0.1;2014-06-22T12:32:35

Logstash config file:  
input { file { path =\> "C:/dashboard/datafiles/users.txt"}}  
filter {grok {match =\> ["message","%{NUMBER:siteid};%{IP:usersip};%{TIMESTAMP\_ISO8601:signdate}"]} geoip {source =\> "usersip"}}  
output { elasticsearch { hosts =\> "127.0.0.1:9202"}}

Now when I config a pattern in Kibana, it doesn't recognize my data correctly. It recognizes the field siteid correct, but then "usersip" is read as "1;127.0.0" and "signdate" as "1;127.0.0.1;20"

I hope someone can help out with this one as I am running out of ideas.

Kind regards, Bo

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [October 21, 2015, 11:08am UTC](https://discuss.elastic.co/t/filter-issue-with-logstash-beta-2-0/32662/2 "2015-10-21T11:08:47Z")

</div>

I can't reproduce what you're seeing.

```
$ cat test.config 
input { stdin {} }
output { stdout { codec => rubydebug } }
filter {
  grok {
    match => [
      "message",
      "%{NUMBER:siteid};%{IP:usersip};%{TIMESTAMP_ISO8601:signdate}"
    ]
  }
}
$ cat data 
1;127.0.0.1;2014-06-22T12:32:35
$ /opt/logstash/bin/logstash -f test.config < data
Logstash startup completed
{
       "message" => "1;127.0.0.1;2014-06-22T12:32:35",
      "@version" => "1",
    "@timestamp" => "2015-10-21T11:07:36.109Z",
          "host" => "lnxolofon",
        "siteid" => "1",
       "usersip" => "127.0.0.1",
      "signdate" => "2014-06-22T12:32:35"
}
Logstash shutdown completed
```

---

<div class="post-metadata">

**Author:** ![boeriksencom](https://avatars.discourse-cdn.com/v4/letter/b/e274bd/32.png) [@boeriksencom](https://discuss.elastic.co/u/boeriksencom)\
**Post date:** [October 21, 2015, 11:28am UTC](https://discuss.elastic.co/t/filter-issue-with-logstash-beta-2-0/32662/3 "2015-10-21T11:28:20Z")

</div>

Could it be an issue with kibana? This is how kibana filters the data. It doesn't recognize signdate as a time-field name when create an index pattern.

 ![](https://us1.discourse-cdn.com/elastic/original/2X/c/c54e99b5a66102493839c6bcae2e04d40cc61538.PNG)

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [October 21, 2015, 11:33am UTC](https://discuss.elastic.co/t/filter-issue-with-logstash-beta-2-0/32662/4 "2015-10-21T11:33:24Z")

</div>

Kibana shows what's in Elasticsearch and Elasticsearch stores what Logstash gives to it. Logstash creates indexes based on the `@timestamp` field which, absent a date filter, will contain the time when the message entered Logstash.

I don't think the document shown in the Kibana screenshot was created by a Logstash instance with the configuration you showed us. I've given you an example that works as expected. Do you get a different result?

---

<div class="post-metadata">

**Author:** ![boeriksencom](https://avatars.discourse-cdn.com/v4/letter/b/e274bd/32.png) [@boeriksencom](https://discuss.elastic.co/u/boeriksencom)\
**Post date:** [October 21, 2015, 12:31pm UTC](https://discuss.elastic.co/t/filter-issue-with-logstash-beta-2-0/32662/5 "2015-10-21T12:31:00Z")

</div>

Excuse me for my ignorance but do you know a way to test like you do using data in the command prompt on a windows machine?

---

<div class="post-metadata">

**Author:** ![boeriksencom](https://avatars.discourse-cdn.com/v4/letter/b/e274bd/32.png) [@boeriksencom](https://discuss.elastic.co/u/boeriksencom)\
**Post date:** [October 21, 2015, 12:32pm UTC](https://discuss.elastic.co/t/filter-issue-with-logstash-beta-2-0/32662/6 "2015-10-21T12:32:50Z")

</div>

My suspicion is that it is related to the txt-file. The same setup though works fine using logstash-2.0.0-beta1 and elasticsearch 2.0.0-beta1

---

<div class="post-metadata">

**Author:** ![boeriksencom](https://avatars.discourse-cdn.com/v4/letter/b/e274bd/32.png) [@boeriksencom](https://discuss.elastic.co/u/boeriksencom)\
**Post date:** [October 21, 2015, 12:41pm UTC](https://discuss.elastic.co/t/filter-issue-with-logstash-beta-2-0/32662/7 "2015-10-21T12:41:15Z")

</div>

..Or of course related to the newer versions of elasticsearch and logstash 🙂

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [October 21, 2015, 2:58pm UTC](https://discuss.elastic.co/t/filter-issue-with-logstash-beta-2-0/32662/8 "2015-10-21T14:58:00Z")

</div>

> Excuse me for my ignorance but do you know a way to test like you do using data in the command prompt on a windows machine?

It works the same way. You won't have the "cat" command and the path to the Logstash executable will be different, obviously.

---

<div class="post-metadata">

**Author:** ![boeriksencom](https://avatars.discourse-cdn.com/v4/letter/b/e274bd/32.png) [@boeriksencom](https://discuss.elastic.co/u/boeriksencom)\
**Post date:** [October 26, 2015, 12:21pm UTC](https://discuss.elastic.co/t/filter-issue-with-logstash-beta-2-0/32662/9 "2015-10-26T12:21:42Z")

</div>

Hi again Magnus. I removed logstash 2.0.0-beta2 and installed current production version 1.5.4 and now it reads my data correctly. So guess that there is a bug in the current beta version of logstash regarding the parsing of txt files.

Thanks for your help

/bo

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:25am UTC](https://discuss.elastic.co/t/filter-issue-with-logstash-beta-2-0/32662/10 "2017-07-06T05:25:23Z")

</div>

Closed on July 6, 2017, 5:25am UTC


