# Filtered Log messages show up as empty fields in Kibana
**URL:** https://discuss.elastic.co/t/filtered-log-messages-show-up-as-empty-fields-in-kibana/315400
**Category:** Logstash
**Created:** [September 28, 2022, 7:17pm UTC](https://discuss.elastic.co/t/filtered-log-messages-show-up-as-empty-fields-in-kibana/315400 "2022-09-28T19:17:25Z")
**Posts on this page:** 9
**Page:** 1
### Author:  [@Chma](https://discuss.elastic.co/u/Chma)
#### Post date: [September 28, 2022, 7:17pm UTC](https://discuss.elastic.co/t/filtered-log-messages-show-up-as-empty-fields-in-kibana/315400/1 "2022-09-28T19:17:25Z")
I have filtered my log message using grok. But when I check Kibana, I find the new fields on the left side of the page, but they are empty. I am also getting the` _grokparsefailure` tag.
Here's an example of my log message:
```auto
[2022-09-28 18:11:25,144] {processor.py:641} INFO - Processing file /opt/airflow/dags/dag_filtered.py for tasks to queue
```
Here's my logstash config file:
```auto
input {
beats {
port => 5044
codec => "line"
}
}
filter{
grok {
match => { "message" => "%{TIMESTAMP_ISO8601:timestamp}]%{DATA:class} %{SPACE}%{LOGLEVEL:loglevel} -%{GREEDYDATA:logMessage}" }
overwrite => ["message"]
}
date {
match => ["timestamp", "MMM dd yyyy HH:mm:ss", "MMM d yyyy HH:mm:ss", "ISO8601"]
target => "@timestamp"
}
}
output {
elasticsearch {
hosts => ["${IP}:9200"]
index =>"logss-%{+YYYY.MM.dd}"
}
}
```
And here's my filebeat configuration:
```auto
filebeat.inputs:
- type: filestream
id: my-filestream-id
enabled: true
paths:
- /home/ubuntu/logs/**/*.log
filebeat.config.modules:
path: /etc/filebeat/modules.d/*.yml
reload.enabled: false
setup.template.settings:
index.number_of_shards: 1
output.logstash:
hosts: ["${ip}:5044"]
processors:
- add_host_metadata:
when.not.contains.tags: forwarded
- add_cloud_metadata: ~
- add_docker_metadata: ~
- add_kubernetes_metadata: ~
- drop_fields:
fields: ["agent", "cloud", "ecs", "host", "input", "tags", "log.offset"]
ignore_missing: true
```
When I test my log message and the grok pattern I have on Grok Debugger, it works fine. So what am I missing?
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [September 28, 2022, 10:11pm UTC](https://discuss.elastic.co/t/filtered-log-messages-show-up-as-empty-fields-in-kibana/315400/2 "2022-09-28T22:11:32Z")
Can you give an example of the [message] field on an event that has a \_grokparsefailure tag?
---
### Author:  [@Chma](https://discuss.elastic.co/u/Chma)
#### Post date: [September 28, 2022, 10:36pm UTC](https://discuss.elastic.co/t/filtered-log-messages-show-up-as-empty-fields-in-kibana/315400/3 "2022-09-28T22:36:03Z")
> [@Badger](#):
>
> \_grokparsefailure
I have shared a couple of them below:
```auto
Traceback (most recent call last)
```
```auto
[2022-09-23 22:51:02,857] {logging_mixin.py:115} INFO - [2022-09-23 22:51:02,857] {dag.py:2379} INFO - Sync 1 DAGs
```
I am guessing that the error is because they have different patterns? I tried to fix this by having multiple match patterns in my logstash configuration(shared below) but I still get the` _groksparse` failure in my tag.
```auto
match => { "message" => ["%{TIMESTAMP_ISO8601:timestamp}%{DATA:class} %{SPACE}%{LOGLEVEL:loglevel} -%{GREEDYDATA:logMessage}", "%{TIMESTAMP_ISO8601:timestamp}%{DATA:class} %{SPACE}%{LOGLEVEL:loglevel} -%{GREEDYDATA:logMessage}, execution_date=%{GREEDYDATA:execution_date}, start_date=%{GREEDYDATA:start_date}, end_date=%{GREEDYDATA:end_date}",
"%{GREEDYDATA:logMessage}" ]}
```
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [September 28, 2022, 11:11pm UTC](https://discuss.elastic.co/t/filtered-log-messages-show-up-as-empty-fields-in-kibana/315400/4 "2022-09-28T23:11:50Z")
I do not get a \_grokparsefailure for either of those. You might want the first pattern to start with `\[%{TIMESTAMP_ISO8601:timestamp}\]`.
---
### Author:  [@Rios](https://discuss.elastic.co/u/Rios)
#### Post date: [September 29, 2022, 6:15am UTC](https://discuss.elastic.co/t/filtered-log-messages-show-up-as-empty-fields-in-kibana/315400/5 "2022-09-29T06:15:32Z")
Add handling for parse failure to know which a line/data cause an error.
You will have the field: "original" in version ELK 8+, or do not overwrite the "message" field.
Avoid GREEDYDATA, use DATA which is faster.
```auto
output {
if "_grokparsefailure" in [tags] {
elasticsearch {
hosts => ["${IP}:9200"]
index =>"logss-%{+YYYY.MM.dd}"
}
# or save in a file
file { path => "/path/grokfailure_%{+YYYY-MM-dd}.txt" }
}
else {
elasticsearch {
hosts => ["${IP}:9200"]
index =>"logss-%{+YYYY.MM.dd}"
}
}
}
```
---
### Author:  [@Chma](https://discuss.elastic.co/u/Chma)
#### Post date: [September 29, 2022, 2:23pm UTC](https://discuss.elastic.co/t/filtered-log-messages-show-up-as-empty-fields-in-kibana/315400/6 "2022-09-29T14:23:54Z")
That's strange, because I am getting \_grokparsefailure for messages without timestamp or Loglevel, and also empty messages.
How can I handle empty messages and messages without timestamp, like these
```auto
message: -------------------------------------------------------------------------------- tags: _grokparsefailure
message: AIRFLOW_CTX_EXECUTION_DATE=2022-09-29T14:04:11.795487+00:00 tags: _grokparsefailure
```
---
### Author:  [@Rios](https://discuss.elastic.co/u/Rios)
#### Post date: [September 29, 2022, 2:47pm UTC](https://discuss.elastic.co/t/filtered-log-messages-show-up-as-empty-fields-in-kibana/315400/7 "2022-09-29T14:47:04Z")
If message starts with ---
`if ([message] =~ /^-{3}/ { ... }`
If message starts with [date]
`if ([message] =~ /^\[\d{4}-\d{2}-\d{2}/ { ... }`
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [September 29, 2022, 5:01pm UTC](https://discuss.elastic.co/t/filtered-log-messages-show-up-as-empty-fields-in-kibana/315400/8 "2022-09-29T17:01:49Z")
> [@Chma](#):
>
> `"%{GREEDYDATA:logMessage}"`
If that is one of your pattern it should always match and you should _never_ get a \_grokparsefailure tag. That suggests that you are not running with the configuration that you think you are.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 27, 2022, 5:02pm UTC](https://discuss.elastic.co/t/filtered-log-messages-show-up-as-empty-fields-in-kibana/315400/9 "2022-10-27T17:02:21Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.