[logstash] Cannot find the logs that donot match the grok filter

Hello, dear all,

I updated the logstash from vesrion 2.1 to 6.4, and I find there are some differences between them which caused my problem.

  1. I cannot find any logs that don't match my grok filters in the kibana for version 6.4, but for the older logstash, I can find them with the tags: "_grokparsefailure ".

  2. I also can't see any error in the /var/log/logstash/logstash-plain.log for the new version logstash.
    And I can find it many _grokparsefailure in the /var/log/logstash/logstatsh.log for the old version logstash.

And for me, I need to save all the logs in the elasticsearch, include the parse error log.

How can I do my job for losgtash v6.4?

Any advice is very appreciated, thanks in advance.


I had found a way to resolve this issue -- add a filter that can match each log.
But it still cannot be seen in the elasticsearch.

And in the end, I found that, I need to extract a field with the type date, or it cann't be indexed to "logstash-YY-MM-DD.log", that's why the log cannot be see in the kibana.

And I am wondering why in the logstash of 2.1 , it can use the default field @timestamp as the elasticsearch index but logstash v6.4 cannot do it as this way.

If anybody knows why, please let me know.

Thanks very much.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.