# Grok pattern is working if the "input" is a file, but not if the "input" is beats
**URL:** https://discuss.elastic.co/t/grok-pattern-is-working-if-the-input-is-a-file-but-not-if-the-input-is-beats/238935
**Category:** Beats
**Tags:** filebeat
**Created:** [June 27, 2020, 8:12am UTC](https://discuss.elastic.co/t/grok-pattern-is-working-if-the-input-is-a-file-but-not-if-the-input-is-beats/238935 "2020-06-27T08:12:09Z")
**Posts on this page:** 4
**Page:** 1
### Author:  [@Aj\_t](https://discuss.elastic.co/u/Aj_t)
#### Post date: [June 27, 2020, 8:12am UTC](https://discuss.elastic.co/t/grok-pattern-is-working-if-the-input-is-a-file-but-not-if-the-input-is-beats/238935/1 "2020-06-27T08:12:09Z")
Sorry about the long post. I have a multi-line CAS log I am shipping via filebeats to logstash running on a different server.
Here's a sample of the original multi-line log:
```
2020-06-21 00:24:00,833 INFO [org.apereo.inspektr.audit.support.Slf4jLoggingAuditTrailManager] - Audit trail record BEGIN
=============================================================
WHO: jane.doe
WHAT: [event=success,timestamp=Sun Jun 21 00:24:00 CDT 2020,source=RankedAuthenticationProviderWebflowEventResolver]
ACTION: AUTHENTICATION_EVENT_TRIGGERED
APPLICATION: CAS
WHEN: Sun Jun 21 00:24:00 CDT 2020
CLIENT IP ADDRESS: 10.10.10.10
SERVER IP ADDRESS: 20.20.20.20
=============================================================
```
I am using `multiline` option in my `filebeat.yml` to handle this. Here's my `filebeat.yml` sample file:
```auto
filebeat.inputs:
- type: log
enabled: true
paths:
- /var/log/cas_sample.log
multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
multiline.negate: true
multiline.match: after
output.logstash:
hosts: ["30.30.30.30:5044"]
```
This results in the following **single line log** entry on the logstash side:
```auto
2020-06-21 00:24:00,833 INFO [org.apereo.inspektr.audit.support.Slf4jLoggingAuditTrailManager] - Audit trail record BEGIN\n=============================================================\nWHO: jane.doe\nWHAT: [event=success,timestamp=Sun Jun 21 00:24:00 CDT 2020,source=RankedAuthenticationProviderWebflowEventResolver]\nACTION: AUTHENTICATION_EVENT_TRIGGERED\nAPPLICATION: CAS\nWHEN: Sun Jun 21 00:24:00 CDT 2020\nCLIENT IP ADDRESS: 10.10.10.10\nSERVER IP ADDRESS: 20.20.20.20\n=============================================================
```
Here's my main logstash pipeline (it's the only one being used):
```auto
input {
beats {
port => "5044"
}
}
filter {
grok {
match => { "message" => "%{NOTSPACE:date} %{TIME:time} .*WHO: %{NOTSPACE:user_name}.*\\nWHAT.*ACTION: %{NOTSPACE:action}.*\\nAPPLICATION.*CLIENT IP ADDRESS: %{IP:source}.*SERVER IP ADDRESS: %{IP:destination}" }
}
remove_field => ["what", "message", "@version", "@timestamp", "path", "host"]
}
output {
stdout { codec => rubydebug }
}
```
But this doesn't do anything to the log. But if I change the pipeline `input` from `beats` to `file` and directly fed the **single line log** to logstash, then it works.
```auto
input {
file {
path => ["/tmp/cas_test.log"]
sincedb_path => "/dev/null"
start_position => "beginning"
}
}
```
And the output would look like this:
```auto
"action" => "AUTHENTICATION_EVENT_TRIGGERED",
"source" => "10.10.10.10",
"destination" => "20.20.20.20",
"date" => "2020-06-21",
"user_name" => "jane.doe",
"time" => "00:24:00,833"
```
So what am I doing wrong? Can anyone tell me why `beats` input isn't working with this grok pattern?
Thanks!
---
### Author:  [@ptamba](https://discuss.elastic.co/u/ptamba)
#### Post date: [June 27, 2020, 1:25pm UTC](https://discuss.elastic.co/t/grok-pattern-is-working-if-the-input-is-a-file-but-not-if-the-input-is-beats/238935/2 "2020-06-27T13:25:24Z")
what does the output look like when you use beats input?
---
### Author:  [@Aj\_t](https://discuss.elastic.co/u/Aj_t)
#### Post date: [June 27, 2020, 6:37pm UTC](https://discuss.elastic.co/t/grok-pattern-is-working-if-the-input-is-a-file-but-not-if-the-input-is-beats/238935/3 "2020-06-27T18:37:24Z")
Here's the output with when I use the `beats` input:
```auto
{
"@version" => "1",
"message" => "2020-06-21 00:24:00,833 INFO [org.apereo.inspektr.audit.support.Slf4jLoggingAuditTrailManager] - Audit trail record BEGIN\n=============================================================\nWHO: jane.doe\nWHAT: [event=success,timestamp=Sun Jun 21 00:24:00 CDT 2020,source=RankedAuthenticationProviderWebflowEventResolver]\nACTION: AUTHENTICATION_EVENT_TRIGGERED\nAPPLICATION: CAS\nWHEN: Sun Jun 21 00:24:00 CDT 2020\nCLIENT IP ADDRESS: 10.10.10.10\nSERVER IP ADDRESS: 20.20.20.20\n=============================================================",
"agent" => {
"type" => "filebeat",
"ephemeral_id" => "SOME_ID",
"id" => "SOME_OTHER_ID",
"hostname" => "SOME_HOST_NAME",
"version" => "7.6.0"
},
"ecs" => {
"version" => "1.4.0"
},
"@timestamp" => 2020-06-27T06:37:02.955Z,
"log" => {
"flags" => [
[0] "multiline"
],
"offset" => 1,
"file" => {
"path" => "/var/log/cas_sample.log"
}
},
"tags" => [
[0] "beats_input_codec_plain_applied",
[1] "_grokparsefailure"
],
"host" => {
"name" => "SOME_HOST_NAME"
},
"input" => {
"type" => "log"
}
}
```
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 25, 2020, 6:37pm UTC](https://discuss.elastic.co/t/grok-pattern-is-working-if-the-input-is-a-file-but-not-if-the-input-is-beats/238935/4 "2020-07-25T18:37:34Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.