# Multiline codec in Logstash 6.6 not working
**URL:** https://discuss.elastic.co/t/multiline-codec-in-logstash-6-6-not-working/172156
**Category:** Logstash
**Created:** [March 13, 2019, 1:23pm UTC](https://discuss.elastic.co/t/multiline-codec-in-logstash-6-6-not-working/172156 "2019-03-13T13:23:21Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@svm89](https://discuss.elastic.co/u/svm89)
#### Post date: [March 13, 2019, 1:23pm UTC](https://discuss.elastic.co/t/multiline-codec-in-logstash-6-6-not-working/172156/1 "2019-03-13T13:23:21Z")
Hi guys,
I am new with the elk stack, but am trying to implement it to get nicer logs for my application.
Here is my config:
```
input {
file {
path => "/var/log/file.log"
codec => multiline {
#pattern => "^\d\d\d\d\/\d\d\/\d\d\s\d\d\:\d\d\:\d\d"
#pattern => "^\d\d\d\d(\/|-)\d\d(\/|-)\d\d(\s|T)\d\d\:\d\d\:\d\d\.\d\d\d"
#pattern => "^\s{24}.*$"
#pattern => "^.*(NFO).*$"
#pattern => "^.*application:\s{15,}.*$"
#pattern => "^.*application:\s{15,}.*$"
pattern => "^\w\w\w\s\d\d\s\d\d\:\d\d\:\d\d.*\|.*$"
negate => "true"
what => "previous"
}
}
}
filter {
if [programname] == "application" {
dissect {
mapping => {
"message" => "%{priority}|%{tag}|%{new_message}"
}
add_field => ["log", "application"]
remove_field => ["message"]
}
mutate {
gsub => ["priority","^$","NULL"]
gsub => ["tag","^$","NULL"]
gsub => ["new_message","^$","NULL"]
strip => ["priority","tag"]
}
}
}
output {
if [type] == "rsyslog" {
elasticsearch {
hosts => ["127.0.0.1:9200"]
}
stdout { codec => rubydebug }
}
}
```
And here are my sample logs:
Mar 13 13:40:02 hostname application: NFO|HTTPpoll7 |HTTP PACKET FROM 127.0.0.1:port TO 127.0.0.1:port
Mar 13 13:40:02 hostname application: GET /status?module=CNX&mode=extended HTTP/1.1
Mar 13 13:40:02 hostname application: Host: 127.0.0.1:port
Mar 13 13:40:02 hostname application: User-Agent: Zend\_Http\_Client
Mar 13 13:40:02 hostname application: NFO|HTTP |HTTP PACKET TO 127.0.0.1:port FROM 127.0.0.1:port
Mar 13 13:40:02 hostname application: HTTP/1.0 200 OK
Mar 13 13:40:02 hostname application: Date: Wed, 13 Mar 2019 12:40:02 GMT
Mar 13 13:40:02 hostname application: content-length: 2049
Mar 13 13:40:02 hostname application: content-type: text/plain; charset=UTF-8
Mar 13 13:40:02 hostname application: NFO|HTTP |End of HTTP connection 127.0.0.1:port
As you can see, I've tried multiple regex for the multiline codec, but nothing seems to work, meaning that the output is the same as the input in Kibana. Or maybe I don't how/where to look for the output.
Please help with any advice.
Thanks a lot!
---
### Author:  [@Ganesh2303](https://discuss.elastic.co/u/Ganesh2303)
#### Post date: [March 13, 2019, 1:38pm UTC](https://discuss.elastic.co/t/multiline-codec-in-logstash-6-6-not-working/172156/2 "2019-03-13T13:38:02Z")
why do you need multiline and your log look like each message look like single line.
---
### Author:  [@svm89](https://discuss.elastic.co/u/svm89)
#### Post date: [March 13, 2019, 2:05pm UTC](https://discuss.elastic.co/t/multiline-codec-in-logstash-6-6-not-working/172156/3 "2019-03-13T14:05:11Z")
There was some trimming when I added the logs. Here is a snip of how they look like:
Ideally, would be something like:
```
Mar 13 13:40:02 hostname application: NFO|HTTPpoll7 |HTTP PACKET FROM 127.0.0.1:port TO 127.0.0.1:port \n GET /status?module=CNX&mode=extended HTTP/1.1 \n Host: 127.0.0.1:port \n User-Agent: Zend_Http_Client
Mar 13 13:40:02 hostname application: NFO|HTTP |HTTP PACKET TO 127.0.0.1:port FROM 127.0.0.1:port \n HTTP/1.0 200 OK \n Date: Wed, 13 Mar 2019 12:40:02 GMT \n content-length: 2049 \n content-type: text/plain; charset=UTF-8
Mar 13 13:40:02 hostname application: NFO|HTTP |End of HTTP connection 127.0.0.1:port
```
Also, I think I need to mention that my "message" field is " NFO|HTTP |End of HTTP connection 127.0.0.1:port" - as an example.
From my point of view, this would be the ideal outcome, but I am open to suggestions.
---
### Author:  [@Ganesh2303](https://discuss.elastic.co/u/Ganesh2303)
#### Post date: [March 13, 2019, 2:30pm UTC](https://discuss.elastic.co/t/multiline-codec-in-logstash-6-6-not-working/172156/4 "2019-03-13T14:30:23Z")
> [@svm89](#):
>
> codec =\> multiline { #pattern =\> "^\d\d\d\d/\d\d/\d\d\s\d\d:\d\d:\d\d" #pattern =\> "^\d\d\d\d(/|-)\d\d(/|-)\d\d(\s|T)\d\d:\d\d:\d\d.\d\d\d" #pattern =\> "^\s{24}._$" #pattern =\> "^._(NFO)._$" #pattern =\> "^.application:\s{15,}.$" #pattern =\> "^.application:\s{15,}.$" pattern =\> "^\w\w\w\s\d\d\s\d\d:\d\d:\d\d._|.\*$" negate =\> "true" what =\> "previous" }
Try this you will get output
codec =\> multiline {
pattern =\> "[a-zA-Z0-9:]+NFO"
negate =\> "true"
what =\> "previous" }
---
### Author:  [@svm89](https://discuss.elastic.co/u/svm89)
#### Post date: [March 13, 2019, 2:40pm UTC](https://discuss.elastic.co/t/multiline-codec-in-logstash-6-6-not-working/172156/5 "2019-03-13T14:40:15Z")
Thank you for the suggestion!
I tried it, but it is the same.
Maybe some more details will help: from the above logs, "Mar 13 13:40:02" gets into the timestamp field, "hostname" gets into the sysloghost field, "application" goes into the programname field and the message field starts from ":". They get automatically mapped like this and I only need to use the multiline codec on the message field. Is this possible?
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [March 13, 2019, 3:50pm UTC](https://discuss.elastic.co/t/multiline-codec-in-logstash-6-6-not-working/172156/6 "2019-03-13T15:50:37Z")
```
codec => multiline { pattern => " NFO\|" what => "previous" negate => true auto_flush_interval => 2 }
```
Once you have gotten the date, hostname, application you can then remove them using
```
mutate { gsub => ["message", "^[A-Za-z]{3}\s[0-9]{1,2} [0-9]{2}:[0-9]{2}:[0-9]{2} [a-zA-Z]+ [a-zA-Z]+: ", "" ] }
```
Which will leave you with
```auto
"message" => "NFO|HTTP |HTTP PACKET TO 127.0.0.1:port FROM 127.0.0.1:port\nHTTP/1.0 200 OK\nDate: Wed, 13 Mar 2019 12:40:02 GMT\ncontent-length: 2049\ncontent-type: text/plain; charset=UTF-8"
```
---
### Author:  [@svm89](https://discuss.elastic.co/u/svm89)
#### Post date: [March 14, 2019, 11:16am UTC](https://discuss.elastic.co/t/multiline-codec-in-logstash-6-6-not-working/172156/7 "2019-03-14T11:16:57Z")
Thank you for the suggestion!
I managed to get the most I could with the aggregate filter.
Thank you all for your input!
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [April 11, 2019, 11:17am UTC](https://discuss.elastic.co/t/multiline-codec-in-logstash-6-6-not-working/172156/8 "2019-04-11T11:17:01Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.