# How to make filebeat multiline plugin send the unmatched logs
**URL:** https://discuss.elastic.co/t/how-to-make-filebeat-multiline-plugin-send-the-unmatched-logs/117906
**Category:** Beats
**Tags:** filebeat
**Created:** [January 31, 2018, 10:02pm UTC](https://discuss.elastic.co/t/how-to-make-filebeat-multiline-plugin-send-the-unmatched-logs/117906 "2018-01-31T22:02:22Z")
**Posts on this page:** 5
**Page:** 1
### Author:  [@saikrishnagaddipati](https://discuss.elastic.co/u/saikrishnagaddipati)
#### Post date: [January 31, 2018, 10:02pm UTC](https://discuss.elastic.co/t/how-to-make-filebeat-multiline-plugin-send-the-unmatched-logs/117906/1 "2018-01-31T22:02:23Z")
Hi, I am using filebeat 5.6.X version
With the below config, filebeat sends five lines to elasticsearch from below log file, using multi line pattern. But the first line " host down true wiley-host" is dropped. How to get the line "host down true wiley-host" into elasticsearch as a separate document. any suggestions??
**Below is the actual log file i am parsing**
host down true wiley-host
Alert high CPU status on host wiley-host
cannot reach the host. Port not reachable
host down form twenty minutes
host status unknown for last twenty minutes
service execution failed with exit status 22
**Below is my filebeat config**
```
filebeat:
prospectors:
- input_type: log
paths:
- "/var/lib/host/wiley.log"
fields:
type: hostlog
fields_under_root: true
multiline.pattern: '(exit status 22$)'
multiline.negate: true
multiline.match: before
multiline.max_lines: 5
output:
kafka:
hosts:
- host1.skg.com:29092
- host2.skg.com:39092
- host3.skg.com:49092
topic: logs-%{[type]}
```
**output in kibana**
message: Alert high CPU status on host wiley-host
cannot reach the host. Port not reachable
host down form twenty minutes
host status unknown for last twenty minutes
service execution failed with exit status 22
**Note** : I am not using the filter pattern on logstash for the reason(below) mentioned in logstash docs
If you are sending multiline events to Logstash, use the options described here to handle multiline events before sending the event data to Logstash. Trying to implement multiline event handling in Logstash (for example, by using the Logstash multiline codec) may result in the mixing of streams and corrupted data.
---
### Author:  [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [February 1, 2018, 8:15am UTC](https://discuss.elastic.co/t/how-to-make-filebeat-multiline-plugin-send-the-unmatched-logs/117906/2 "2018-02-01T08:15:17Z")
What do you mean by 'dropped'?
Why do you set `max_lines: 5`? The clips the multiline event after 5 lines. Lines afterwards, but still missing the pattern will not be send.
Do you have a more complate log file? With messages before/after?
---
### Author:  [@saikrishnagaddipati](https://discuss.elastic.co/u/saikrishnagaddipati)
#### Post date: [February 2, 2018, 5:59pm UTC](https://discuss.elastic.co/t/how-to-make-filebeat-multiline-plugin-send-the-unmatched-logs/117906/3 "2018-02-02T17:59:15Z")
@steffens
**What do you mean by 'dropped'?**
I want the first line of the log file "host down true wiley-host" also to be sent to elasticsearch and view the message in kibana as a new document.
**Why do you set max\_lines: 5?**
I want to store the matching line that is "service execution failed with exit status 22" and the four lines above it as a single document.
Below is the log file with before and after
```
host down true wiley-host
Alert high CPU status on host wiley-host
cannot reach the host. Port not reachable
host down form twenty minutes
host status unknown for last twenty minutes
service execution failed with exit status 22
retrying to execute service "ping-check"....
"ping-check" failed with exit status 22
```
Are you saying that the lines missing the pattern will not be send to elasticsearch?
---
### Author:  [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [February 8, 2018, 11:22pm UTC](https://discuss.elastic.co/t/how-to-make-filebeat-multiline-plugin-send-the-unmatched-logs/117906/4 "2018-02-08T23:22:40Z")
This is not how max\_lines works. One configures one pattern and depending on `negate` setting, all lines (not) matching will be combined into one event. While matching multiple lines until flush, only the first `max_lines` will be published. The first `host down true wiley-host` should still be shipped, but not dropped.
I assume you don't have any other lines before or after your file. That is basically one file per status check. Plus I assume you want to generate 2 events:
1.
```auto
host down true wiley-host
Alert high CPU status on host wiley-host
cannot reach the host. Port not reachable
host down form twenty minutes
host status unknown for last twenty minutes
service execution failed with exit status 22
```
and
1.
```auto
retrying to execute service "ping-check"....
"ping-check" failed with exit status 22
```
Using these multiline settings:
```auto
multiline.pattern: 'exit status \d+$'
multiline.negate: true
multiline.match: before
```
works well for me:
```auto
{
"@timestamp": "2018-02-08T23:20:06.934Z",
"@metadata": {
"beat": "filebeat",
"type": "doc",
"version": "7.0.0-alpha1"
},
"message": "host down true wiley-host\nAlert high CPU status on host wiley-host\ncannot reach the host. Port not reachable\nhost down form twenty minutes\nhost status unknown for last twenty minutes\nservice execution failed with exit status 22",
"source": "/tmp/testinput.txt",
"offset": 228,
"prospector": {
"type": "log"
},
"input": {
"type": "log"
},
"beat": {
"name": "xxx",
"hostname": "xxx",
"version": "7.0.0-alpha1"
}
}
{
"@timestamp": "2018-02-08T23:20:06.934Z",
"@metadata": {
"beat": "filebeat",
"type": "doc",
"version": "7.0.0-alpha1"
},
"message": "retrying to execute service \"ping-check\"....\n\"ping-check\" failed with exit status 22",
"input": {
"type": "log"
},
"prospector": {
"type": "log"
},
"beat": {
"name": "xxx",
"hostname": "xxx",
"version": "7.0.0-alpha1"
},
"source": "/tmp/testinput.txt",
"offset": 313
}
```
Just do not use `max_lines: 5`.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 8, 2018, 11:22pm UTC](https://discuss.elastic.co/t/how-to-make-filebeat-multiline-plugin-send-the-unmatched-logs/117906/5 "2018-03-08T23:22:51Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.