# Logstash showing only parse failures from log

**URL:** <https://discuss.elastic.co/t/logstash-showing-only-parse-failures-from-log/62800>\
**Category:** Logstash\
**Created:** [October 12, 2016, 11:06am UTC](https://discuss.elastic.co/t/logstash-showing-only-parse-failures-from-log/62800 "2016-10-12T11:06:23Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![ddelaria](https://avatars.discourse-cdn.com/v4/letter/d/8491ac/32.png) [@ddelaria](https://discuss.elastic.co/u/ddelaria)\
**Post date:** [October 12, 2016, 11:06am UTC](https://discuss.elastic.co/t/logstash-showing-only-parse-failures-from-log/62800/1 "2016-10-12T11:06:23Z")

</div>

Hi All,  
not sure what's happening on my stack, but yesterday I've included a new log file on the filebeat config on a couple of client machines.  
Everything was working fine, but including this new log file I'm experiencing a really weird behaviour: the only messages showing up in Kibana are the ones with tags:\_grokparsefailure (about 1% of the log entries).  
The rest of the log file is simply not showing up...

## On the client

On the client I've got filebeat 1.2.2, here's the filebeat.yml extract I've added:

> ```
> paths:
> - /mypath/mylog.log
> document_type:
> MY_TYPE
> input_type: log
> -
> 
> ```

## On the ELK server

Elasticsearch 2.4.1  
Logstash 2.2.4  
New file in /etc/logstash/conf.d:

> filter {  
> if [type] == "MY\_TYPE" {  
> grok {  
> match =\> { 'message' =\> ['%{TIME:my\_timestamp}%{SPACE}%{WORD:Severity}%{SPACE}[%{DATA:Thread}]%{SPACE}(%{DATA:my\_Component})%{SPACE}%{GREEDYDATA:my\_Message}' ] }  
> }  
> date {  
> match =\> ["my\_timestamp", "HH:mm:ss,SSS"]  
> timezone =\> ["Europe/London"]  
> }  
> }  
> }

Did any of you ever face this kind of issue?

Best regards!

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [October 12, 2016, 11:14am UTC](https://discuss.elastic.co/t/logstash-showing-only-parse-failures-from-log/62800/2 "2016-10-12T11:14:33Z")

</div>

Wild guess: The new events you're trying to send are incompatible with the mappings in Elasticsearch. If that's the case you should find details in the Logstash log.

---

<div class="post-metadata">

**Author:** ![ddelaria](https://avatars.discourse-cdn.com/v4/letter/d/8491ac/32.png) [@ddelaria](https://discuss.elastic.co/u/ddelaria)\
**Post date:** [October 12, 2016, 12:23pm UTC](https://discuss.elastic.co/t/logstash-showing-only-parse-failures-from-log/62800/3 "2016-10-12T12:23:45Z")

</div>

Tthanks for the quick feedback, I've given it a look but nothing is being trapped in there.  
Any suggestion on how to increase its verbosity?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [October 12, 2016, 12:59pm UTC](https://discuss.elastic.co/t/logstash-showing-only-parse-failures-from-log/62800/4 "2016-10-12T12:59:24Z")

</div>

I'd expect that to be logged with the default log level, but starting Logstash with `--verbose` or even `--debug` will increase the log level.

---

<div class="post-metadata">

**Author:** ![ddelaria](https://avatars.discourse-cdn.com/v4/letter/d/8491ac/32.png) [@ddelaria](https://discuss.elastic.co/u/ddelaria)\
**Post date:** [October 12, 2016, 1:33pm UTC](https://discuss.elastic.co/t/logstash-showing-only-parse-failures-from-log/62800/5 "2016-10-12T13:33:33Z")

</div>

Ok, there's something when running in `--debug`, I can see the messages incoming so at least I am now 100% sure filebeat has no guilt in this.  
Yet, I'm not seeing any error trapped, it doesn't complain anywhere about mappings in elasticsearch, I'm even seeing the assignment of data to fields (even if reading the debug output is quite dispersive).  
Should I look for some specific string which may look harmless at a first sight?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:34am UTC](https://discuss.elastic.co/t/logstash-showing-only-parse-failures-from-log/62800/6 "2017-07-06T04:34:33Z")

</div>